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INTRODUCTION 
1.1 Purpose of Procedural Inventory 

, The primary purpose of this Volume is to present in one coltection 
a systematic inventory of research and analytic procedures appropriate 
for generating information on knowledge production, diffusion, and 
utilisation, Our main concern has been with those procedures that focus 
on the utilization of knowledge by policymakers and practitioner^. 

As an area of scientific inquiry and professional practice "knowledge 

at* 

use" has grown in scope and importance over the last ten years. Prac- 
titioners and academics alike have come to recognize the urgency of 
examining the knowledge-bases which underlie policies and Dfrograms in 
the realms of public (governmental) activities as well as private business. 
While this interest has sprung from—and been applied to— many substantive 
areas, the field of education has been a particularly rich source of 
Information and Insights. Educators have had a long and fruitful history 
of experimentation and Innovation, as they have attempted to Improve the 
processes whereby the cumulative growth of knowledge is passed from 
generation to generation. 

Attempts over the last ten years to specify, measure, and assess 
various aspects of the knowledge production and use have become in- 
creasingly numerous. In general, such studies have evidenced wide 
variations 1s purpose, structure, analytic focus, and theoretical 
underpinnings. In large measure, this variation may be understood by 
considering knowledge utilization as an emerging field of study in its 
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own right. As defined* by Mull 'ins and others, 1 emerging study areas— 
those whidrbegin to focus on- a common set of problems and over time begin 
to share a. common set of assumptions and operational techniques—tend in 
early stages Of , their development; to be characterized by unclear termin- 

. ology and a tendency for those working in the area to* regard it as "fluid, 

° 2 - 

witlvshifting boundaries." Thus, while knowledge use- is a field vJhose 

substance and "boundaries are likely to^solidify in coming years, there 

is today considerable ambiguity in its core assumptions and goals. 3 For 

this reason, among others, it seems to us imperative to assess research 

V 

procedures in a systematic and coherent way. This volume represents, 

as far as we are aware, one or the few attempts to address this task. 

/ 

1.2 Sampling Procedures 

* * J * 

The original objective for this phase of the project was to focus 

on research instruments— more or less carefully calibrated procedures 
designed to measure properties whose meanings, have been defined in ad- 
vance by researchers. Early in the project, however, it became clear that 
an exclusive focus on this objective would eliminate a great number of 

research procedures, particularly in the area of qualitative methodology, 

/ 



Nicholas Mullins. Theory and Theory Group Construction . (New Yorfc: -Harper 
and Row, 1973. 

2 « "^^^^ 

Anthony, L.H., H. East and M. Slater, "The Growth of the Literature of 
Physics," Reports on Progress in Physics , 32 (1969) :731. 

3 

^ Xarol Weiss has characterized the fiel4 as conceptually "soggy" , ( Using 

SocTfri Research in Public Policy Making ; Lexington, MA: D.C. Heath, 1§77:11) 
Nel son^nd Winter refer to the Tl tangled literature 4 ' of the field which makes 
it difficult to arrive at criteria for defining knowledge use as a dependent 
variable. See R.R. Nelson and S.G. Winter, "In Search of Useful Theory of 
Innovation." Research Policy , 6, 1(1977); andfflerald Zaltman "Construing 
Knowledge Use,' 1 Volume II of this report. 
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which are important elements of the content and evolution of the field. 

The sampling procedure used to select methods and techniques grew 
put of consultations with members of our national advisory network which 
includes leading experts on knowledge use. In addition, searches were 
made of the Social Science Periodical Index and the Social Science 
Citation Index. We a] so examined existing bibili graphical on knowledge 
transfer and planned change compiled by the National Institute of Mental 
Health in conjunction with the Human Interaction Research Institute. 
Finally, a computerized bibliographic search was conducted through the 

■r' 

Social Sciences Information Utilization Laboratory at the University of 
Pittsburgh. This search included items which appeared between 1969 and the 
present in the Current Index to Journals in Education. 4 

As initial sampling began to identify procedures for potential in- 
cjlusion, we utilized snowball sampling to contact additional researchers. . 
The final number of candidate procedures was approximately 200, of which 
a small number were eliminated because the^were dated or because there was 
not sufficient information to describe them to others. 

1.3 Format for Procedures 

The documentary and archival search, together with inquiries to 
investigators, yielded a pool of some two-hundred candidate procedures. 
Our aim was to select- from this pool a subset with maximum relevance^to 
methodological problems addressed in the project. The original set of 
candidate procedures was eventually narrowed to sixty- four procedures 
.that were compared, contrasted, and assessed in accordance with a 
standardized abstracting procedure. Each abstract provides a general 

3 

CUE is produced by the Education Resources Information Center of the 
National Institute of Education. 



information profile including author, availability, purpose, variables, 
description, development^ reliability/validity, and administration. 

1-4 Format and Criteria^for* Study Profiles 

In addition, we wished to investigate the s£udy designs in terms of 
which procedures were employed. For this purpose we constructed, 
where possible, a study profile 6 describing the unit of analysis, sampling 
procedure, type of design, research methods, practice area or population, 
definition of knowledge use, and other key methodological characteristics. 

Study profiles were almost entirely limited to major studies of knowledge use 
identified in the course of the project, since 1t was less manageable to 
examine the characteristics of study designs which yielded new or promising* 
procedures (e.g., cognitive mapping techniques) but which were not expressly 
directed at the conceptualization and measurement of knowledge use. 
General information profiles were limited to procedures that satisfied two 
criteria: reproducibility and face relevance to knowledge use. 

The criteria of reproducibility required that a given procedure be 
sufficiently specific,, regular, and orderly that Its steps may be repeated 
by some other investigator. By applying this criterion of Inclusion we 
were forced to abandon a large number of procedures, in some Instances— 
especially those involving qualitative methods—researchers have explicitly 
challenged or disavowed the appropriateness of reproducibility as a criterion. 

The second criterion— face relevance to the study of knowledge use- 
required judgments about the conceptualization of knowledge and Its uses. 

3 : — t — 

See Volume I, Appendix B. 

6 See Volume I, Appendix C. 
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Here we were guided by general considerations that procedures should 
permit the acquisition of information about multiple dimensions of know- 
ledge use; that procedures should be appropriate for* investigating alter- 
native theories; and that procedures should somehow elicit information 
about cognitive properties, since it "is this feature that provides the 
field with a unique purpose and identity. The criterion of face relevance, 
when applied to research on knowledge use proper, resulted in the inclusion 
of some fifty-five procedures. Additional procedures were included as 
relevant, even though they had not been developed or used by researchers 
who consider themselves part of the field. ' 
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INTERACTION PROCESS ANALYSIS 



AUTHOR: 



AVAILABILITY: 



Robert F. Bales 



Bales, Robert F. Personality and Human Behavior , New York: 
Holt, Rinehart and Winston, 1970. 



PURPOSE : The Interaction Process Analysis fs a, system for assessing 

H ... basic structure characteristics and dynamic processes 

[found] in small groups " (p. 110). 

VARIABLES : The observer coding system measures twelve variables in 

group processes: solidarity, tension release, Agreement 

Giving Suggestion, giving opinion, asking for orientation, 
asking for opinion, Asking for Suggestion, Disagreement, 
Showing Tension, and Showing Antagonism. 

» 

DESCRIPTION : Coding Sheets with each of the twelve variables listed 

— * above including explicit descriptions are used to obs6rve 

and categorize behavior of individuals in group settings. 

DEVELOPMENT : Robert Bales first developed this system and subsequent 

analytic routines in the late 1940's in an effort "to 

develop a context-free instrument" with which to study 
attitudes and behaviors of individuals in group settings. 



RELIABILITY/ 
VALIDITY: 



Test-retest reliability of .41 for a median significant 
r was reported in three-five person situations. 

Construct validity was obtained for several variables 
by Jewell, Lewis and Withall (1961). Hamblin and Miller 
(1960) found the twelve variables to be highly inter- 
correlated. 



ADMINISTRATION: 



Several methods of administration are available— Mechanical 
recording devices, or manual scoring forms being the most 
prevelant. Bales suggested the latter being utilized by 
observers behind orne^ay glass. Two scorers are suggested. 
Each variable listed fcbove is numbered, each actor is 
assigned a number, and directions of interactions between 
actors are noted. Provisionals made for types pf inter- 
actions directed at self or others outside the group. 
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SOURCES : Bales, R.F. Personality and Interpersonal Behavior . New 
York: Holt, Rinehart and Winston, 1970. 

Bales, R:F. and H. Gerbrands, "The Interaction Recorder: 
An Apparatus and Checklist for Sequential Content Analysis 
of Social Interaction." Human Relations . 1948, 1, 456-463. 

Newell, J.M., W.W. Le»1s, and J. Withall. "Use of a 
Communication Model to Study Classroom Interactions". 

- Paper read at the annual meeting of the American 
Educational Research Association, 1961. 
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STUDY PROFILE 

OIMENSION , DESCRIPTION ( COOE 

Unit of Analysis M} groups 01 02 ©04 05 06 07 08 09 

Sampling..... N/A 01 02 03 04 07 @09 

' gjgj^ Q *. 03 04 05 06 07 08 09 

Research Methods structural observation 01 ©° 3 04 05 06 07 08 09 

Analytic Methods ratings/codes 1 (?) 3 4 5 8 9 

Analytic Focus non-qual1tat1ve < * 

Reliability reported 1 © 8 

* 

Validity. „. reported 1 (5 8 

p .< ; 

Definition Of Use., . „ 1 2 (z) 8 9 

symbolic \U 

Object of Use Qther 12 345(7)89 

Practice Area and 

P °P ulation General 
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SURVEY ON EVALUATION STUDIES OF SOCIAL ACTION PROGRAMS 



AUTHOR: 



AVAILABILITY: 



Ilene N. Berstein and Howard E?; Freeman 

I 

Bernstein, I.N., and H.E. Freeman. 
preneurial Research . New York, NY: 
tlon, 1975. 



Academic and Entre- 
Russell Sage Founda- 



PURPOSE: 



The survey was designed to gather information on the scope 
of 4 evaluation research system: funding of evaluation re- 
search, the organizations which carry out the research and 
methods employed. The researchers, using "generally 
accepted" criteria of social science research quality, 
attempted to determine the conditions under which high 
and low quality evaluation research occur. 



VARIABLES : The survey was composed of seven major sections: (1) 
characteristics of personnel conducting the research, 
(2) characteristics of the organization conducting the 
research, (3) characteristics of the target population, 
C4) information on the theory, goals and nature of the 
program belna evaluated, (5) information on methodological 
procedures, (6} information on procedures employed to mea- 
sure impact, and (7) information on planned activities for 
dissemination and utilization of findings, 



DESCRIPTION: 



382 surveys were identified as evaluation studies by granting 
agencies or other sources. 318 surveys were returned, of 
which eighty-two were not identified by respondents as being 
evaluation studies. 236 surveys were used in analysis, ini- 
tial surveys were sent to evaluations of programs in health, 
education, welfare, manpower, income security, housing and 
public safety. 



DEVELOPMENT: 



Not reported. 



RELIABILITY/ 
.. VALIDITY: 



No reliability or validity reported. 



ADMINISTRATION: 



For each of the studies included in the sample, a mailed 
self-administered questionnaire was sent to the study direc- 
tor along with a letter explaining the nature and purpose 
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of the study. Persons who .did not respond were contacted 
by telephone. This procedure was followed until approxi- 
mately eighty-three percent of the questionnaires were 
returned. 
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STUDY PROFILE 

DIMENSION DESCRIPTION CODE 

Unit of Analysis Small group 01 02@04 05 06 07 08 09 

Sampling Purposive - 01@03 04 07 08 09 

Quasi -Experimental without ^— n 

Design...... control groups and/or 01 02 03 04(05^06 07 08 09 

randomization 

Research Methods Questionnaires 01 02 03 04 05(0§)07 08 09 

Analytic Methods N/A 1 2 3 4 5 ® 9 

Analytic Focus Non-qualitative i 1 @ 8 9 

Reliability Not reported 1 (f)\8 

Validity.... Not rep orted 1 ® 8 

Definition of Use Uncodafale 1 2 3 8 ® 

; Object of Use.. Evaluation ©2345689 

j ■ 

Practice Area and 

Population , Evaluation studies, practitioners. 
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IMPACT OF THERAPEUTIC EFFECTIVENESS DATA ON CMHG MANAGEMENT 



AUTHOR- 



Douglas A. Blgelow and James A. Ciarlo 



AVAILABILITY: 



Bigelow, DJ\. and J. A. Ciarlo. "The Impact of Therapeutic 
Effectiveness t)ata on Comnunlty Mental Health Center * 
Management" in G.V. Glass Evaluation Studies Review Annual , 
Vol. I. Beverly HiTls, CA: Sage, 19757" 



PURPOSE : The purpose of this study 1s to provide empirical data 

regarding the commonly held assumption of comnunlty 

mental health evaluation, that 1s, that data fed back 
to center managers have an Impact on the management, result- 
ing 1n optimization of programs. 1 

VARIABLES : This study attempted to jneasure the baseline levels of 

— = various Inputs in which management 1s embedded (e.g., 

structure of prlortles, political pressures, funding 
considerations, etc.); the relevance of evaluation 
criteria to managerial concerns; and a preliminary Indication 
of the Impact of feedback Information on management. 

DESCRIPTION : Two different questionnaires were used 1n this study. 

The Decision-Making Questionnaire consisted of seven groups 

of questions. The first group was Intended to establish 
the current importance of each af twelve types of data that 
may have been available tcr the respondent as a program 
manager. The second group dealt with competing Influences 
and pressures. The third, fourth and fifth groups of 
questions attempted to assess the decisionmaking structure 
of the* respondent (I.e., priorities, strategies of approach 
and perception of the problem). The sixth group of questions 
asked the respondent to rate the Importance of thirteen 
mental health objectives from his own point of view on 
a ten point scale. The seventh question was open-ended, 
giving the respondents a chance to propose additional 
criteria. 

The Information Utilization Questionnaire was an open-ended 
questionnaire consisting of four questions regarding the 
use of specific body of data bearing on a program manage- \ 
ment 1ssue(375-76) . \ 



DEVELOPMENT: 



Not reported. 
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RELIABILITY/ 

VALIDITY : Not reported. 



ADMINISTRATION : Three strategies were used 1n the study. The major, one 

Involved Interviewing supervisors using an Information 

Use and Decision-Making Questionnaire. The purpose of the 
'study was outlined; the investigator read the questions, 
gave any necessary explanations, and recorded the responses. 

> 

A second strategy was to interview the supervisors when a 
substantial program modification had been made within 
their jurisdiction. The Interviewing procedure was Identical 
to that above, but a slightly modified form of the question- 
naire was employed. Three such interviews were conducted. 

The third strategy of assessing decision making 1n the 
center was to introduce a body of data bearing on a program 
management Issue during a meeting of supervisors arjd 
then to trace the effecU over a two-week period. When the 
data had been presented, the managers were asked about the 
Implications of the information and what they Intended 
to do on the strength of it. Then, three weeks later, the 
supervisors were again contacted by telephone and asked several 
questions contained 1n a Data Utilization Questionnaire. 
Two such incidents were studied, with seven supervisors 
participating. 
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STUDY PROFILE 

DIMENSION DESCRIPTION CODE 

Unit of Analysis 0ther (conmunlty Mental Health 01 02 03 04 0 5 06© 08 09 

Center managers) w 

Sampling Convenience 02 03 04 07 08 09 



Design SSStaSSon" 11 '* 1 01 02 ° 3 ^© 06 07 08 09 

Research Methods interview 01 02 03 04 05 (25) 07 08 09 

Analytic Methods Ratinqs 

1 03 4 5 8 9 

Analytic Focus Non-qualitative i 1 (£) 8 9 

Reliability.;. I... L. i. Not reported (T) 2 8 

Validity • Not reported (Q 2 8 

Definition of Use Instrumental 

Qz 3 8 9 

Object of Use Uncodable 1 2 3 4 5 6 8 (O 

Practice Area and 

Population Mental Health, Program Administrators 
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LABORATORY VERSUS LECTURE IN TRAINING EXECUTIVES 



Lee Bolman 



Bolman, Lee. "Laboratory Versus Lecture in Training 
Executi ves , " The Journal of Applied Behavioral Science , 
July-September"7 T l97{r " 



The study focuses on what it does when used as one. of two 
alternative methods of increasing the competence of 
business executives to deal with interpersonal phenomena. 



The variables measured were Self Perception or Personal 
Behavior in Groups (measured along two dimensions: 

1) ability to deal with emotions in a group setting; 

2) openness to *1rrformat1i>n and influence from others), Beliefs 
about effective behavior (or Managerial Behavior), Managerial 
Experience and Behavior Scores. 



Each variable is measured by a questionnaire with different 
items. The Analysis of Personal Behavior in Groups was 
measured by two scales. 1) APB feelings scale , which is a 
measure of ability to cope with emotions and consists of 
participants self-rating on ability to express feelings, 
tendency to seek close relations with others, tolerance of 
the expression of affection and antagonism; 2) APB openness 
scale , consists of following items: ability to listen with 
understanding to others, willingness to tie Influenced 
by others, open to comments about own behavior, openness to 
opinions opposed to one's own. 

The Managerial Behavior Questionnaire presented the respond- 
ent with several hypothetical situations which might arise 
in" a business setting and asked him to indicate in a few 
sentences how he would handle them. 

The Managerial Experiences Questionnaire has a series 
of multiple-choice items, asking the respondent as to 
what he thought would be the most effective way to behave 
in a given situation. 

The Behavior Scores wer* obtained from tape recordings 
of T-Group meetings and small group discussion of about 
twelve members without a leader. * 
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Example of the Analysis of Personal Behavilr 1n Groups 1s as 
folldws: \ 

1. Ability to listen to others 1n an understanding way 



0 . 1 2 3 4 5 6 7 
Low ' High 

2. Tendency to build on the previous Ideas of other 
group members 



0 1 
Infrequent 



Frequent 



DEVELOPMENT : Twd seniors of a unverslty program for busfhess executives 

— which Included laboratory training were compared with 

two seniors of the same program 1n which the topic of 
Interpersonal behavior was treated through lectures, 
case discussions, and readings. All the participants 1n 
the, program were male. > 



RELIABILITY/ 
VALIDITY: 



SOURCES: 



The test-retest reliability coefficients were .69 for 
the feelings scale and .77 for the openness scale. 

Bolman, Lee. "Laboratory Versus Lecture 1n Training ■ 
Executl ves . " Journal "of Applied Behavioral Science , 
{^September, 1970, 3?33Z5T ~ 
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STUDY" PROFILE 



DIMENSION DESCRIPTION CODE 

Unit of Analysis 01 02 03 04 05 06 07 Qfi 09 

• Individual - x T 

- — / 

Sampling 01 02 03 04 07 08 09 

Purposive 

_ . Quasi -Experimental without M M M nt A _ nr . nn ,, 

Desljn control groups and/or random . 01 02 03 04 05 06 07 08 09 1( 

izalinn - ■ fl 

Research Methods... a 01 02 03 04 05 06 07 08 09 . 

questionnaires 

Analytic Methods ratings 1 2 3 * 5 8 9 



' Ana,yt1c FoeuS non-qualitative \ 128 

.,*rt1*11«y nmtai 12 8 

Validity ... N/A 12 8 

0ef1n1t1on of Use. Instrumental • 1 2 3 

Object of Use........... person-embodied 123 

Practice Area and. 

Population Education Research, Business Executives 
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OPEN-ENDED PERCEIVED CHANGE; MEASURE 



Douglas R. Bunker, Eric S. Knowles, Matthew B. Mjjes. 

Bunker, Douglas R. "Open-Ended Perceived Change Measure".* 
Program 1n Policy Sciences, Amherst, New York: State 
University of New York at Buffalo f 

The Open-Ended Perceived Change Measure was originally 
designed to assess perceived changes 1n Interpersonal and 
work-related behavior, as these are affected by laboratory 
human relations training. Bunker's form measures "Impact" 
of training on interpersonal and work-related behavior 1n 
seventeen categories. 

The test measures the following seventeen variables: 

A. Overt Operational Changes B. Inferred Changes 1n 

Insight and Attitude 

1. Communication • 1. Awareness of human behavior 

a. sending (express feelings, 2. Sensitivity to group 
etCi) behavior 

b. receiving (listens, etc) 3. Sensitivity to people 

c. unspecified (communicates 4. Acceptance of other people 
better, etc.) 5. Tolerance of new Information 

2. Relational facility 6. Self-confidence 

3. Risk taking 7. Comfort 

4. Increased Interdependence 8. Insight Into self and role 

5. Functional flexibility 

6. Self-control 

• G. Global judgements 

The following open-end question 1s reacted to 1n writing 
by subjects: 

Over a period of time, people may change 1n the ways 
that they work with other people. Sinc e 
[month of. workshop or training experience being 
evaluated], do you feel you have changed your behavior 
1n working with people 1n any specific ways, as 
compared with the previou s [ year, academic 

year, or other meaningful pejrlodj? 

YES . NO 



29 
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If yes, please describe. 

A parallel form of the Instrument is reacted to by a total 
of five or six "descrlbers", persons who associate with the 
subject. They are asked, "... has he changed his behavior.. 

DEVELOPMENT : Tfrls method of retrospective change measure using job 

associates reports was originally developed by Buchanan 

(1957a, 1957b), for evaluating supervisory training. . 
Miles (1965) created th,e present measure in a study of 
thirty- four elementary school principals, plus two control 
groups of n-29'and 148, and produced a preliminary scoring 
manual. Bunker (1965) and Bunker and Knowles (1967) in- 
ductively developed the content-analytic categories, on a 
sample of 346 persons attending six training laboratories. 
' After coder reliability checks, the scheme was revised 
to its present form. 

Moscow (1969) translated the Bunker scheme and coding 
* manual into Dutch, and found in a study of fifty- two 

T-group participants and thirty-one controls that 
"categories Bl and B3 should be collapsed because of 
cojier unreliability. 



RELIABILITY/ 
• VALIDITY: 



No test-retest reliability -data are available on the re- 
sponses themselves. -Coder, reliability (agreement) was 
achieved at or beyond the 90% level by Bunker (1965) on 
his revised scheme; Moscow (1969) obtained 88% agreement 
on the unitizing of changes, and 82% on the assignment of 
changes to categories. Coder stability over. time (un- 
specified) was found by Moscow to be 86%; Bunker foUnd 
90% stability in coding. after an eighteen-month ^interval. 

Content validity seems well extabllshed. Miles (1965) in art 
eight-month follow-up found that 73% of those principals 

\ attending a training laboratory showed change over a "base 
rate" of 2, using his scheme; only 17% and 29% of control 

^groups did so. 



ADMINISTRATION: 



This query is sent by mail to persons having participated 
in the training or treatment experience to be evaluated, 
as well as to members of any control group being used. 
Each subject is also asked to nominate eight to ten people 
with whom he is in regular working contact on the job, 
and who are in a position to comment on his behavior. The 
instrument is self-administering, and probably takes less 
than five minutes for most subjects to complete. 
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In the Miles Scoring scheme, each separate change reported 
1s given one point, and verified, changes— those listed by 
subject and his descrlbers—receive an additional point. 
Non-socially desirable changes are similarly scored and 
given a minus sign. Total ^scores 1s the sum of all ^points. 

In the BunKer-Knowles scoring scheme, changes are coded 
in the seventeen categories (and indicated as positive or 
negative). A total score is computed for the "self" 
descriptions* and another for the total number of discrete 
changes mentioned by descrlbers. A more conservative 
verified change score is the number of changes in which 
two or more descrlbers concur. 

SOURCES : Bunker, B. "Individual Applications of Laboratory Training." 
Journal of Applied Behavioral Science , 3(1967) :505-23. 

Bunker, D. and E. Knowles. "Comparison of Behavioral 
Changes Resulting from Human Relations Training 
Laboratories of Different Lengths"., Journal of Applied 
Behavioral Science , 3(1967) : 505-23. ~ 

\ Miles, M,B./* Changes During and Following Laboratory 

i Training: A Clinical Experimental Study." J ournal 

of Applied Behavioral Science , 1(1965) :215-4Z 
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A ' 

STUDY PROFILE 

t 

DIMENSION DESCRIPTION ^ CODE ^_ 

ft 

Unit of Analysis Individual 01 02 03 04 05 06 07 08 09 

\ ' . r : 

Sampling. Purposive 01 02 03 04 °* 08 09 

Design '.. 01 02 03 04 05 06 07 08 09 10 



Research Methods Open-ended questionnaire . 01 02 03 04 05 06 07 

Analytic Methods other 1 2 3 4 5 8 9 

' <! 

Analytic Focus Qualitative 1 1 , 2 8 9 



Reliability reported 

-Validity reported 



Definition of Use instrumental 



Object of Use Person-embodied Innovations 12 345 68 9 



Practice, Area and 

Population Evaluation of normal adults, Psychotherapy, Normal 

adult, and mental patients 



9 
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DESIRE FOR CERTAINTY TEST 



AUTHOR: 



O.G. Brim, Jr. 



AVAILABILITY: 



Brim, O.G., Jr. "Attitude Content— Intensity and Probability 
Expectations." American Sociological Review. Vol. 20, (1955) 
68-76. 



PURPOSE : The scale was constructed to measure individual differences 
in motivation for certainty. 



VARIABLES: 



'Desire for certainty", which 1s closely related to 
'Intolerance for ambiguity". 



DESCRIPTION: 



The Desire for Certainty Test comprises thirty-two statements 
about everyday events; e.g., "Ttie chances that American 
citizens will believe in God are about in 100." 

Respondents place a probability value in the blank space 1n 
each sentence. In addition, they Indicate their confidence 
in each .estimate by rating it on a five-point certainty 
scale from 1 (very sure) to 5 (not sure at all). 

The statements included four from each of eight different 
subject-matter areas: education, recreation, politics, 
economics, religion, health, family, transportation and 
communication. For e.g.: 

1. \jhe chance that an adult American will earn at least 

$4000 a yefar are about in 100. 

2. The chance that a student earning a law degree will 
quit before getting the degree are about _in 100. 

3. The chances that a sexual pervert will have a low 
intelligence are about in 100. 



4. The chances that a couple ge 
will later divorce are 



itJMjuj ma 
J|n 100. 



married this year 



5. The chances that a governor of a state will be elected 
for a second term in office are about in 100. 



DEVELOPMENT : Not reported. 
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RELIABILITY/ 
VALIDITY: 



ADMINISTRATION: 



SOURCES: 



The te'st was administered on college students, all of whom 
were enrolled in Introductory sociology courses. For one 
group of students (n=50), a corrected split-half reliability 
coefficient of .81 was obtained. 

For 500 students the test scores "appeared" normally dis- 
tributed. A study by Brim and Hoff (1$57) offers construct 
validity for the scale. "Desire for certainty" was found to 
be significantly related to extreme responding on several 
measures, including a variety of attitude scales. 



No time estimate was mentioned by Brim, but the test would 
probably take about twenty minutes for self-admini strati on. 

Scoring is based on the assumption that a strong desire 
for certainty will be expressed by two tendencies: 1) 
to select probability values close to the extremes of zero 
to 100; and 2) to express confidence in these extreme 
choices. Thus the authors took the distance of each 
probability extlmate from its nearest end point (0 or 100) 
and multiplied this value by its certainty score (very 
sure ■ 1; not sure at all ■ 5). The ( products were then 
summed over all thirty- two test items. By this procedure 
a low score indicates a high desire for certainty. 



Brim, 0. and D. Hoff. "Individual and Situational V 
Differences in Desire for Certainty." Journal of 
Abnormal and Social Psychology . Vol. 54, (19577:225-29. 

Frenkel-Brunswik, Else. "Intolerance of Ambiguity as an 
Emotional and Perceptual Personality Variable." Journal 
of Personal ity ; Vol, 18, (1949) : 108-43. 

Frenkel-Brunswik, Else. "Personality Theory and Perception." 
In R.R. Blake and G.V. Ramsey (eds.), Perception : An 
Approach to Personality . New York: Ronald, (1951)1756-419. 
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STUDY PROFILE 

DIMENSION ; DESCRIPTION CODE 

Unit of Analysis Individual (oi) 02 03 04 05 06 07 08 09 

& . 

Sampling.. Purposive Ql^)o3 04 07 08 09 

Design expllcl^theSry 6 01® 03 04 05 06 07 08 09 

Research Methods ...content Analysis 01 02 ®° 4 05 V 07 08 09 

Analytic Methods Ratings 10 3 4 5 8 9 

Analytic Focus Non-qualitat1ve i 1 ® 8 9 

Reliability ( Reported l ® 8 

VaH<"ty Reported 108- 

Definition of Use Instrumental (?) 2 3 8 9 

Object of Use Person-embodied innovations 1 2 3 4 / (T)6 8 9 

Practice Area and 

Population Psychiatry, Psychology and Mental health, Students in 

College 
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THE USE OF SOCIAL SCIENCE KNOWLEDGE IN POLICY DECISIONS 
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AUTHOR: 



Nathan Cap! an, Andrea Morrsion and Russel J. Stambaugh 



AVAILABILITY: 



Caplan, N., A." Morrison and R.J. Stambaugh.. The Use of 
Social Science Knowledge In. Policy Decisions at the" 
^atlonaTTeyel . Ann Arbor: University of M1ch~1gan, 
Center for Research on Utilization of Scientific Knowledge. 



PURPOSE: The purposes of this study were first to provide empirical 

— data on the prevailing state of social science research 

utilization among government officials; second, to determln 
ways to facilitate the use of research 1n order to Improve 
ongoing government activities.* 



VARIABLES : The questions 1n the Interview could be categorized Into 

six major areas of 1nou4ry: |L) Respondent's awareness of 

available relevant socialised ence Information; 2) Re- 
spondent's sel f-reportWdAisVof social science knowledge 
1n policy relevant s1tuat1fifns?v3) Respondent's evaluation 
of the worth and perceived objectivity of various kinds 
of data, data collection methods and measurement 
procedures; 4) Respondent's Interest 1n the development of 
social Indicators as potential scientific measurement of the 
quality of life; 5) Respondent's attitudes about the use 
of social research and his beliefs about factors that In- 
fluence the use of such knowledge 1n policy formulation; 
6) Respondent's personal and educational background, employ- 
ment history, and future career plans. 



DESCRIPTION : The Interview Instrument contained a range of open-and- 

close-ended questions dealing directly and Indirectly 

with social science utilization. 



DEVELOPMENT: 



Not reported. 



> 



RELIABILITY/ 
VALIDITY; 



ADMINISTRATION: 



Not reported. 



The Interviews were conducted wtth 204 Individuals holding 
Important positions 1n various departments, major 
agencies and Commissions of the executive branch of govern- 
ment. 
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The interviews were carried out on a face-to-face basis 
and were conducted by professional Interviewers^ The 
av r?i e u time re( 1 uired for each interview was about one and 
a half hours. The Interviews were tape recorded for 
purposes of editing, and completing the written narrative 
on the interview form. 
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STUDY PROFILE 

DIMENSION DESCRIPTION CODE 

ft * 

Unit of Analysis other (government officials) 01 02 03 04 05 06^))08 09 

'02 03 04 07 08 09 

01 02 03 04 05 06 07 08© 

01 02 03 04 05(06>)7/08 09 



• Convenience ( 


• Uncodable 


. Interview 




. Ratings 


< 


. Non-qualitative, \ 


, Not reported 








Not reported 


Instrumental 


• Uncodable 




^ 




• Federal Policymakers 









T 

1 Q) 3 4 5 8 9 

Qz 8* 

\ 



(T)2 8 
(l)2 3 8 9 
1 2 3 4 5 6 8 0 

^ : 

Practice Area and 
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METHODOLOGICAL PREREQUISITES FOR PSYCHOTHERAPY OUTCOME RESEARCH 



AUTHOR: 



AVAILABILITY: 



\ 

Lawrence Cohei^ 

Cohen, LWence. "Methodological Prerequisites for 
Psychotherapy Outcome Research: A Survey of Clinical 
Psychologists." Knowledge: Creation. Diffusion. Ut1- 
Hzat1on Vol. 2, number 2, (December l980):263-272 



PURPOSE: 



The survey was undertaken to determine the characteristics 
of psychotherapy research which affects Its use 1n prac- 
tice. A second goal was to determine the criteria which 
practitioners rely on when evaluating the adequacy and' 
persuasiveness of psychotherapy research. 



VARIABLES : Variables examined 1n the survey were 1) Characteristics 

Related to Clinical Relevance, 2) Characteristics Relifel 

to Experimental Design, 3) Characteristics Related to^r 
Criteria Measurement, 4) the Relationship of the Study t6 
Previous Research, and 5) .Miscellaneous Characteristics 
* ("research sponsored by an academic Institution", "researcher 
has clinical experience.") «' 



DESCRIPTION : Two types of questionnaires were Included 1n the research. 

— The first consisted of a checklist of twenty four desirable 

characteristics of research, compiled by the researcher. 
Respondents were asked to Indicate characteristics they 
considered to be Important ("crucially Important" and 
"extremely Important".) The second format was a question- 
naire using dpen-ended responses. Respondents were 
1 asked to describe five or fewer "crucial characteristics" 
and five or fewer "extremely Important" characteristics. 



DEVELOPMENT : The survey was constructed on the basis of previous 

' research concerning the qualities of mental health research 

which had a positive or negative Impact on Its usefulness. 
Concerned about d1ssat1sfat1on with "poor linkage between 
applied research and service delivery", the researcher 
attempted to discover the causes of that dissatisfaction, 
and by Implication, to suggest ways 1n which the usefulness 
of such research might be Improved. 




' RELIABILITY/ 
VALIDITY: 



No reliability or validity coefficients were reported. 



24 



ADMINISTRATION: 



The mailed questionnaires were stent to ninety five academic 
„cl1n1c1ans, and fifty private practicing psychologists. 
Packets of questionnaires were also sent to ten Veterans . 
Administration Hospitals, eleven medical schools, ten 
•state mental hospitals, and ten federally funded community 
mental heal th centers . • 

% Results of the research indicated the following characteris- 
tics were of importance. 



RANK ORDER OF IMPORTANT CHARACTERISTICS 
OF PSYCHOTHERAPY OUTCOME RESEARCH 





CHECKLIST 






tSSAY 




1. 


control group 


85 


1.' 


good experimental design 


51 


2. 


follow up measurement 76 


2 


control group 


44 


3. 


appropriate statistic 72 


3. 


general izable patients 


43 


4. 


random assignment to 
groups' 




4. 


appropriate statistics 


34 


5. 


general izable thera- 
pists 


61 


5. 


good criterion measure 


33 


6. 


general izable 
patienjs 


61 


6. 


clinically relevant , 


30 


7. 


an in vivo* study 


59 


X? ■' 


follow-up measurement 


28 


8. 


several dependent 
measures 


48 


8. 


structural characteristics 27 


9. 


large number of 
treatment sites 


37 


9. 


large sample size 


27 


10. 


low level of 
reactivity 


31 


, 10. 


description of treatment 


26 



SOURCES: 



Cohen, Laurence and Kathryn Suchy. "The Bias in Psycho- 
therapy Research 1r» Evaluation." Journal of Clinical 
Psychology, Vol . '35", number' 1 .(January 1979) : 184-87. 



9 

ERIC 



40 



"Factors Affecting the Utilization of Mental 

Health Evaluation Research Finding." Professional Psy- 
chology , November 1977:526-34. 

/ 

— "Clinical Psychologists' Judgements of the 

Scientific Merit and Clinical Relevance of Psychotherapy 
Outcome Research." Journal of Consulting and Clinical 
' 27 (l$?9):42?-237 



Psychology . Vol. 47, number 



6 
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DIMENSION 



STUDY PROFILE 



DESCRIPTION - 



CODE 



Unit of Analysis. 

Sampling..* 

Design 



Individual 



Purposive 



Research Methods. 



Questionnaires 



Analytic Methods .. Bivariate Analysis 



Analytic Focus tan-qualitative 



Reliability. Not reported 



Validity Not reported 

Definition of Use 

Object of Use 



Instrumental 



Evaluation 



Practice Area and ' i 
Population....... 



lental Health, Practitioners 



©02 03 04 05 06 07 08 09 
01 (02)03 04 07 08 09 



Quasi -experimental without 
control groups and/or random 
iaati o n : — 



01 02 03 04(05J06 07 08 09 



01 02 03 04 05 08 09 

1 2(^)4 5 8 9 

T 

l(2)8 9 



© 

0 



2 8 



2 8 



^T)2 3 8 9 




1 2 345689 
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AVAILABILITY: 



PURPOSE: 



VARIABLES: 



DESCRIPTION: 



DEVELOPMENT: 



EDUCATIONAL EVALUATION "UTILIZATION QUESTIONNAIRE 



Barbara Dickey 



Dickey, B. "Utilization of Evaluation of Small-scale 
Innovative Educational Projects," Educational Evaluation 
and Policy Analysis . Vol. 2, Number 6 (November-December, 



The purpose of this study was to examine educational 
evaluation utilization in terms of reported level of 
Use and to identify and measure factors related to the 
reported level of use. 



The study measured the attitude of the decisionmaker toward 
the use of the evaluation, the degree to which the decision- 
maker was Involved 1n the evaluation process, and the re- 
ported level of utilization of the project evaluation. 



The level of utilization was measured by developing a Guttman 
scale from three questions 1n the project director's ques- • 
tionnalre: 

(al How would you rate the usefulness of the evaluation 
process? 

Cbl How would you rate the, usefulness to you of the evaluation 
report itself? 

(cl Thinking about the overall Impact of the evaluation, how 

would you rate its usefulness? 
The respondents were Instructed to rate the evaluations on a 
scale ranging from "very or quite useful" to "limited useful- 
ness or not useful." 



During the fiscal years of 1975 and 1976, sixty projects 1n 
the state of Minnesota received Title IV-C funds. Of these 
sixty projects, fifty-four filed final evaluation reports 
with the State Department of Education. All project directors 
who filed final reports were sent a questionnaire. 




VALIDITY: 



None reported'. 
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ADMINISTRATION: A mail questionnaire was sent to the fifty-four project 

— directors of Title IV-C projects 1n Minnesota who filed 

final reports w,1th the State Department of Education during 
fiscal years 1975-1976. The sample for this study consists 
of forty-seven projects for which there was available both 
a copy of the evaluation report and a returned and completed 
questionnaire. 
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STUDY PROFILE 

v 

DIMENSION DESCRIPTION . . CODE 

Unit of Analysts other, (decision makers) 01 02 03 04 05 06© 08 09 

Sampling census 01 02 03 ® 07 08 09 

a - , 

Design Uncodable . 01 02 03 04 05 06 07 08® 

Research Methods, Questionnaire 01 02 03 04 05 ® 07 08 09 

Analytic Methods Blvariate Analysis 1 2 Q 4 5 8 9 

Analytic Focus Nonqualltative i 1 © 8 9 

Reliability None reported / (\) 2- 8 

■/ 

\ Validity Nor »e reported Q 2 8 

Definition of Use Instrumental (J) 2 3 8 9 } 

Object of Use Evaluation Reports ©2345689 

. 1 , 

Practice Area and c . .. , 

Population Ed u cat '° n evaluation reports 
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WORK ACTIVITIES AND RESEARCH jiiSZ 



Frances B. Dickraan 



Dickraan, F.B. "Work Activities, Settings, Methodologies 
and Perceptions." Knowledge : Creation , Diffusion , Utili- 
zation . Vol. 2> Number 3 (March, 1981): 3/5-87. 



This study is part of a larger study that was conducted to 
examine the nature of applied social research, including 
evaluation research, and its correlates with the organiza- 
tional settings in which it is practiced (378). 



Utilization was measured using a construct consisting of four 
items designed to assess respondents' perceptions of the 
influences their work had that resulted' in recommendations 
for different types of protfrararaajtic changes.. In addition, 
the instrument measured methodological sophistication con- 
sisting of items assessing analytical style, use of sampling, 
type of sampling, etc: The instrument also measured metho- 
dological inconsistencies. as a dummy variable based on the 
methodological sophistication items. > 



Data were collected using a self-administered mail questionnaire. 
The instrument was composed of open-ended and fixed choice items. 



This study is a further exploration of studies by Weeks (1979), 
Useera and DiMaggio (1978) and Van de Vail and Bolas (1977) which 
all examined the relationship ^between methodological variables 
and research utilization. 



Ten percent of the respondents were randomly selected and inter- 
viewed in order to assess the instrument's reliability and va- 
lidity. 



This study was conducted in two counties in Southern California. 
The instrument was a self-administered mail questionnaire. Re- 
spondents were located in three organizational settings most 
likely to employ applied social researchers: (1) social pro- 
grams (e.g., planning administrative and regulatory agencies in 
the two counties); (2) academic institutions within the two 
counties; (3) all entrepreneurial research organizations in 
the two counties. 
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SOURCES : Dlckraan, F.B. "On the Interaction Between Organizational 

Affiliation and Professional Role: A Case Study of 

' Applied Social Researchers." Doctoral dissertation, 

Program 1n Social Ecology, University of California 

at Irvine, 1979. 

Useera, M. and P. D1Magg1o. "An Example of Evaluation Re- 
search as a Cottage Industry." Sociological Methods 
and Research , Vol. 7 (1978): S5-W. 

Weeks, E.C. "Factors Affecting the Utilization of Evaluation 
Findings 1n Administrative Decision-Marking," Doctoral 
dissertation, Program 1n Social Ecology, University of 
California at Irvine, 1979. 

Van de Vail, M. and C. Bolas. "Policy Research as an Agent 
of Planned Social Intervention," Sociolo gical Practice, 
Vol. 2 (1979): 77-95. 
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' STUDY PROFILE 

DIMENSION , DESCRIPTION CODE 

Unit of Analysis Individual @ 02 03 04 05 06 07 08 09 

s 

Sampling..... Census 01 02 03@ 07 08 09 

Design Uncodable 01 02 03 04 05 06 07 08 @ 

Rasearch Methods Questionnaire 01 02 03 04 05(06)07 08 09 

Analytic Methods Multivariate 1 2 3 ©5 8 9 

Analytic Focus Nonqual1tat1ve i 1 © 8 9 

Reliability Not reported Q 2 8 

Valley Not rep0 rted © 2 8 

Definition of Use Instrumental ® 2 3 8 9 

Object of Use....- other 12345(?)89 

Practice Area and 

Population Applied social researchers 

■ 

/ 

?r - '48 
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STUDY 1 OF RESEARCH USEFULNESS 



AUTHOR: 



AVAILABILITY: 



Mary Jo Dukes 



Dukes, M.J., Graduate School of Public and International 
Affairs, University of Pittsburgh, Pittsburgh, PA 15260. 



PURPOSE: The purpose of this study was to determine the characteristics 

of mental health research studies that make them useful 1n 

, work related activities for mental health professionals. 



VARIABLES: 



DESCRIPTION: 



The variables measured can be loosely grouped Into six 
categories: 1) General attitude toward mental health 
research; 2} Relevance of the research to clinical 
practice; 3) Perceived overall usefulness; 4) orientation 
to action/Implementation; 5) perceived research quality; 
6) general demographics. * 



This research 1s a methodological study concerned with 
comparing alternative approaches for Identifying the factors 
that affect mental health practitioners' use of formal 
knowledge— specifically social science research knowledge— 
1n work related activities. The study compares two 
different methodological approaches. 

The first approach 1s a modification of multl attribute 
utility analysis. Thfse data were collected through a 
self-adm1n1stered mall questionnaire 1n which respondents 
were asked to rate two research abstracts on a f1ve-po1nt 
scale Indicating the extent to which a study based on the 
supplied abstract contains nineteen research characteristics. 

The second approach Involves data generation through the 
case of a semi -structured Interview technique, Kelly's 
Repertory Grid. 



DEVELOPMENT : The self-adm1n1stered questionnaire used 1n this study was 

a modification of the Interview used 1n the study conducted 

by Weiss and Bucuvalas (1980). 



RELIABILITY/ 
VALIDITY: 



Not reported. 
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ADMINISTRATION ; The mall questionnaire was mlled to 905 Individuals in 
Pennsylvania Mental Health Base Service Units who provide 

at least some direct mental health service. Four hundred 

eighteen individuals ,res ponded. 

After a preliminary analysis of the mall questionnaire 
data, a subsample of 30 of the respondents were contacted 
and pajcsonally' Interviewed using a modification of Kelly's 
ReperfWy Grid Method. 

SOURCE : Weiss, C.H. and M.J. Bucuvalas. Social Science Research 

and Peel si on Making . New York: Columbia University 

Press, 1980. 

Kelly, G.A. Psychology of Personal Constructs . Vol. 1 and 2. 
W.W. Norton, New York: 1955. 
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STUDY PROFILE 



DIMENSION DESCRIPTION 

ft 

Unit of Analysis 

Individual 

Sampling...... 

Convenience 

0es1 9 n * ' Uncodable ■ 

Research Methods.. Multiple Methods 

Analytic Methods •••Multivariate Analysis 

'K 

Analytic Focus •Qualitative 

Reliability m 

Validity -Not reported 

Definition of Use conceptual » 

Object of Use. other (absftr^sapk 

Practice Area and 

Population Mental health, Practitioners 



X, 




OTpZ 03 Of 05\06 07 08 09 
(oi)02 03 04 07 08 09 



01 02 03 04 05 0^ 07 08(^) 
01 02 03 04 05 01(07 08 09 



1 2 3 05/8 9 
\(l)z 8 ^ 

.0 2 8 



© 



2 8 

1.(0 3 8 9 
1 2 3 4 SI 9 



® 



v 

\ 
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CASE SURVEY OF PLANNED ORGANIZATIONAL CHANGE 



William N. Dunn and Fredric W. Swlerczek. 



Dunn, W.N. and F.W. Swierczek, "Planned Organizational 
Change: Toward Grounded Theory". The Journal of Applied 
Behavioral Science, Vo l. 13, number~2Tl977 ) : 135^T57T 



The purpose of this study was to generate theories of 
knowledge utilization and platined change which are grounded 
1n the professional experiences of practicing social 
"attetttiSts and policymakers. ,>»•• 

* ' \ 

i 

In analysis the following Independent variables were 
employed: 

1. Type of Organization (economlc/serv Ice/commonweal) * 

2. Societal Type (modern/modernizing) 

3. Task Environment (long- term .stable/ long- term .unstable/' 
short-term, stable/short-term, unstable) 

4. Change-Agent Origin (1nd1gehous-1nternal/1nd1genous- 
external/non-1nd1genous-1ntemaT/non1nd1genous-external ) 

5. Mode of Intervention (unilateral /subordinate/ 
del egatl ve/col 1 aboratl ve) 

6. Chwge-Agent Orientation (participative/process 
eng1 neerl ng/expert ) 

7. Origin of Change (1ntemal/external/superord1nate) 

8. Focus of Change (managers/Staff /f1r$t-Hne, supervisors/ 
Une/multlple levels) 

9. Focus of Solution (human/technologlcal/structural/ 
;task/m1xed) 

10. Locus of Change (total organ1zat1on/d1vl£lon/department 
section) 

11. Standardized Strategy (Organizational Development/ 
Participative Management/Sod otechhlcal , Design/ 
Sodoorganlzattonal, Design/Institution, Building/ 
Other) 

12. Methods (untr1ed/proven/single/mult1ple) 

Dependent variables were effectiveness of the change effort 
and degree of adoption of change. f 



The Instrument attempted to synthesize a range of variables 
from existing studies. The Instrument was developed with 
the primary aim of enabling Investigators to code and 
content analyze reported case studies. This same 
Instrument 1s also flexible enough for adaptation 1n the 
form of protocols, Interview schedules, or questionnaires. 
The format employed for coding and content analysis Include: 
eighty variables. 
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DEVELOPMENT: 



The authors reviewed alternative applications of retrospective 
case analysis, which provided a point of departure for de- 
veloping a research Instrument with which to analyze 
reported cases of planned organizational change. Building 
upon past efforts in the area, the authors chose to apply 
content analysis to a theoretical sample of sixty-seven 
cases. 



RELIABILITY/ ! 
VALIDITY: 



Not reported. 



ADMINISTRATION: 



The theoretical (purposive) sample was selected on the basis 
of several explicit criteria: 1) type of organization 
(economic, service, commonweal); 2) societal type (modern 
and modernizing societies); 3) reported outcome of change 
(successful and unsuccessful). The sample design originally 
envisioned 150 cases stratified according to the ttfree 
variables described above. In two successive reviews 
this number was reduced to 100, and then to 67 reportable 
cases, mainly because a large number of cases either contained 
excessive missing observations or were not sufficiently 
descriptive of planned change to justify further analysis. 
No cases occurring before 1945 were analyzed, and as wide 
a cross section as possible of standardized change strate- 
gies (e.g., Organization Development, Institution Building, 
Organizational Design) were Included. 
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STUDY PROFILE 

DIMENSION DESCRIPTION CODE 

i 

% 

Unit of Analysis... uncodable 01 02 03 04 05 06 9 7 J 8 09 

Sampling • Purposive 01 02 03 04 07 08 09 

Design... Uncodable 01 02 03 04 05 06 07 08 09 

Research Methods Analys1s 01 02 03 04 05 06 07 08 09 

Analytic Methods........ B 1var1ate 123458 9 & 

Analytic Focus Non _ qua i 1tat1ve i 1 2 8 9 . 

Reliability Not-Hreported ' 128 

Validity No t reported 1 2 8 

Definition of Use Instrumental 1 2 3 8 9 

Object of Use study " 1*2345689 

Practice Area and 

Population Planned Organization Change, Case studies 
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NURSE'S PROFESSIONAL ATTITUDE SCALE 



39, 



AUTHOR: 



AlfKvJ.T. Fingerhut 



AVAILABILITY: 



Fingerhut, Alfred T. "Nurse's Professional Attitude Scale". 
Martinez, California: Veteran Administration Medical Center. 



PURPOSE : The instrument was designed to determine the degree of 
professional attitudes among staff nurses. 



VARIABLES: 



Attitudes toward professionalism and professional char- 
acteristics in nursing. Professionalism is defined as 
certain specified factors or traits identified in 
literature as being characteristic of a profession; pro- 
fessional attitude is defined as a respondent's positive 
response to the factors or traits identified in literature 
as being characteristic of a profession. 



DESCRIPTION : The instrument consists of thirty, five-point (strongly 

agree — strongly disagree) rating-scale-types that 

address issues of professionalism in nursing. 

Agrfee No: 'Disagree Strongly 
* Opinion Disagree 



Strongly 

. Agree 



1 



1. An RN is able 
to make inde- 
pendent deci- . 
sions concern- 
ing his/her 
assignment. 

*£. There is a feel- 
ing of colleague- 
ship among 
nurses. 

3. A professional 
delegates tasks 
to appropriate 
others . 
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DEVELOPMENT; 



40 



The*Hnstrument was designed to determine the degree of 
professional attitudes among three groups of staff nurses 
who had earned a baccalaureate degree, an associate 
degree, or a diploma In nursing. A review of literature 
.relevant to professionalism provided the basis for the 
items. The Instrument was administered to 25 nurses at 
each of three general hospitals within the same geographic 
region. Based on Item discrimination data (type of 
coefficient not reported), some items were revised. 



RELIABILITY/ 
VALIDITY: 



Using data from the Instrument development study, a 
reliability coefficient of .83 for the professional 
Items and .85 for the nonprofessional items was calculated. 
(Type of coefficient was not reported). The source of the 
Items and the procedure used for development ensure some 
degree of content validity. 



ADMINISTRATION: 



Self -administered; the time reared ffr completion of the 
Instrument was not specified, although 10-15 minutes would 
appear to be adequate. Fingerhut calculated separate scores 
for the "professional" attitude Items and the "nonpro- 
fessional" attitude Items as well as a total score. The 
values assigned to each response 1s Indicated at the top 
of the answer columns on the Instrument. The maximum 
positive score is 150 Indicative of a high professional 
Orientation; the maximum negative score possible 1s 30 
Indicating a low professional orientation; a score of 
90 indicates a neutral orientation. 



SOURCES: 



F1 ngerhu t , Al f red T . A Study of Potential Differentiation 

of Professional Attitudes Between Staff Nurses In 

Selected Veteran AfliiTnlstrat ion HosWEalsT TJnpuiTfshed 
doctoral dissertation, Laurence University, 1977. 
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STUDY PROFILE - ", ..' 

DIMENSION DESCRIPTION CODE 

Unit of Analysis i ndivi<jual 4 > (gl)o2 03 04 05 06 07 08 09 

Sampling Purposive 0l(g)03 04 07 08 OS 

Docian • Quasi -experimental without n , M M aa/CNa* «•» nn 

065190 ' .control groups and/or random- 01 02 03 < °*(05)06 07 08 09 

i nation ■ — 

Research Methods Questionnaire 01 02 03 04 05(06)07 08 09 

Analytic Methods ^Rating 1 (*) 3 4 5 8 9 

Analytic Focus •• Non-qual1tat1ve 1 1 (D 8 9 

> * i 

Reliability ••••Reported , 108 

Validity Not reported (T)2 8 

Definition of Use'. Instrumental (T) 2 3 8 9 

Object or Use study 1 (7)3 4 5 6 8 9 

Practice Area and 

Population Health, Nurses 
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KNOWLEDGE USE AND LOCAL SCHOOL IMPROVEMENT 



AUTHOR: 



William Firestone, et al. 



AVAILABILITY: 



Research for' Better Schools, Inc., 444 North Third Street, 
Philadelphia, PA 19123 



PURPOSE: 



The purpose of this study was to explore the process ~<of the 
transfer of educational research into instructional practice 
through educational development. 



VARIABLES : * Knowledge transfer process was examined at each of four 
stages: 1) the components' development of an approach 
(basic skills or career education); 2) the linker pre- 
sentation of that approach; 3) the school or classroom 
use of that approach; 4) how feedback* from the sites 
was used to modify the approaches. 



DESCRIPTION: 



This was* a longitudinal study of five schools working with 
two development efforts— basic skills and career education. 
Each effort or approach was intended to transfer resources 
knowledge and expertise from literature in a content area 
in order to help schools develop school improvement programs. 



DEVELOPMENT: 



RELIABILITY 
VALIDIT 




Not reported. 



Not reported. 



ADMINISTRATION: 



For two years, field workers from the KBS Field Studies 
Component -studied the school improvement projects of the 
RBS Development Division using qualitative or naturalistic 
field methods. These field workers observed project 
meetings between RBS representatives and participants 
from eight different school districts'. Formal and In- 
formal interviews were conducted with participants in the 
projects from RBS, district offices and schools. Each 
field worker concentrated his/her efforts on one or two 
schools. 
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STUDY PROFILE . 

DIMENSION DESCRIPTION ± CODE 

* % 

Unit of Analysis Organization . 01 02 03 ® 05 06 07 08 09 j 

Sampling Purposive 01@03 04 07 08 09 

a i 

• atiasr^gy ®° 2 « « « « « « « 

Research Methods Hultlple methods Or 02 03 04 OS 06® 08 09 

Anelytlc Methods ^S^^^ © 2 3 4 5 8 9 ' 

Analytic Focus Qualitative i@2 8 9 

Reliability © 2 8 

Not reported 

Validity Not reported ©2 8 

Definition of Use Instrumental © 2 3 8 9 

Object of Use.. Educational Innovation 1 2 Q) 4 5 6 8 9 

Practice Area and 

Population Five schools 
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REGIONAL EDUCATIONAL SERVICE AGENCIES 



William Firestone, et al. 



Research for Better Schools. Annual Report on Study of 
Regional Educational Service Agencies : Fiscal" Year 138>1 . 



The purpose of this study was to determine ways that school 
districts use knowledge for school Improvement and to 
Investigate the relationship between the school districts 
and the regional service agencies (RESA). 



This study examined several aspects of the linkage process: 
1) the type of linkage (technical and political); 2) the 
role of the field agent; 3) the environment of RESAs. 



In conducting this study, the authors were Interested 1n 
four major questions: 

(1) In what ways do school districts use knowledge for 
school Improvement? j 

(2) What 1s the relationship between school district and 
* Its staff and, th6 regional educational service 

agencies (RESA) who provide knowledge? 

(3) What are some Important local schooj district characteristics? 

(4) What are the benefits/drawbacks derived from the school 
d1str1ct-RESA relationship? 



This study builds on previous research On dissemination 
systems. A number of earlier studies have Indicated that 
1ntterorgan1zat1onal arrangements that promote contact between 
local practitioners and external agents promote educational 
change and contribute to knowledge. use 1n school districts. 
Most of these studies have been evaluations of federal 
projects, many of mhlch have been temporary systems. If 
such 1nterorgan1zat1onal arrangements are to become more 
permanent, they will have to takea different form. This 
study 1s helping to determine the extent to which RESAs cur- 
rently provide such arrangements and are likely to in the 
future. 



Not reported. 



ADMINISTRATION; 



Data have been collected 1n two waves. The first wave 
consisted of site visits to twenty-three of the fifty- four 
RESAs 1n the two states: eleven I Us (Intermediate Units), 
two EICs (Educational Improvement Centers), and ten 
County Offices. During site visits, Interview and survey 
data were collected from agency administrators, external 
informants, and field- agents. In each agency data were 
collected from one or two administrators and from three 
to twelve field staff. Complete data were obtained from 
138 field agents, forty-one administrators, and th1rty-s1x 
external Informants— I.e., school superintendents and 
RESA board members. 

A second wave of data collection yielded Information from 
seventy- two school districts. At the district level In- 
terviews were conducted with the person most familiar 
with the overall working relationships between the school 
district and the RESA. Interviews were also conducted 
with two district-wide administrators, two principals, and 
four teachers who were also familiar with the services 
offered by the RESA. These people completed questionnaires 
This procedure produced seventy- two interviews and 537 
usable surveys. (4-5) 
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STUDY PROFILE 

DIMENSION DESCRIPTION CODE 

Unit of Analysis 01 02 03/04>05 06 07 08 09 

Organisation 

— c 

Sam P 11n 9 Purjposive "(S/ 03 04 07 08 09 ^ 

Design...... ^oSs^ WUh0Ut stat1stica1 01 Q2@QA 05 06 07 08 09 

Research Methods Multiple 01 02 03 04 05 06®08 09 

Analytic Methods -unavailable 1 2 3 4 5 © 9 

Analytic Focus Non-Qualitative i l(S) 8 9 

Reliability ^ ported Q) 2 * 

Validity Not reported CD 2 8 

Definition of Use.. Instrumental (7)2 3 8 9 

Object of Use other 1 2 3 4 5 (£j 8 9 

Practice Area and 

Population i... Education, Field agents and school personnel 

t 
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POLICYMAKERS' USE OF EDUCATIONAL RESEARCH 



47 



AUTHOR: 



David H. Florlo, Michael M. Behnmann, and Diane L. Goltz 



AVAILABILITY: 



Florlo, D.H., Behrmann, M.M. and D.L. Goltz. "What Do 
Policymakers Think of Educational Research and Evaluation? 
or Do They?" Educational Evaluation and Policy Analysis, 
Vol. 1, number 6(1W$) :6l-37: 



PURPOSE : The purpose of this study was to Investigate the sources of 

Information used by congressional staff members and to 

determine the relative Importance and value of educational 
research 1n various aspects of the policy process. 

VARIABLES : The study looked at the types of Information used, the 
stage of the legislative process in which different types 

of Information were most useful, the Information needs of 

the respondents. 



DESCRIPTION: 



The interviewees were provided with a set of open-ended 
questions designed to minimize the Imposition of Interviewer 
Influence on their responses. The Interview questions were 
designed to gain a general understanding of (a) .the value 
and use of different types of educational studies, for 
example, evaluation and policy assessment studi*^. research 
studies, systematic data gathering, and concepts social 
problems from social science Inquiry; (b) when, In- the 
different stages of the legislative process (development, 
deliberation, decision, and oversight), various forms of 
Inquiry are most useful; (c) types of Information needed 
but not provided or problems with Information overload; 
(d) that sources provide the most valid and reliable In- 
formation; and (e) how educational Inquiry rates 1n relation 
to other Influences on legislative policy. Interviews were 
taped and reviewed by at least two interviewees for accuracy 
and completeness. 

Most of the Interviewees completed the survey instrument 
Immediately following the Interview. The survey was 
divided Into three parts. First, respondents were asked 
to rank the relative Importance of five general categories 
of policy relevant Information: 1) financial considerations, 
2) Impact (effects and outcomes on Individuals and organiza- 
tions), 3) demographic data on populations served and affected, 
4) Information on policies and programs. Second, subjects 
were asked to provide priority rankings of different kinds 
of information within each of the general categories; and 
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third, they were asked to Identify the five most Important 
specific types of Information fromvthe forty-one items listed 
under the five categories. - (63-64) x 




DEVELOPMENT : It islmportant to know where the congressional staff 

get their knowledge and Information given their important 

role 1n the legislative process. It is useful for researchers 
and policy analysts to understand the degree to which 
congressional staff value and use the results of education- 
al research and evaluation. Jn addition, 1t is useful to 
understand the barriers that block such use. 



RELIABILITY/ 
VALIDITY: 



Not reported. 



ADMINISTRATION: 



Twenty-six members of the congressional staff (personal 
and conmrittee/subcoirnrlttee) dealing with educational 
legislation were interviewed concerning the sources of 
their information and the relative Importance of educational 
Inquiry in various stages of the policy process. 

Twenty-four staffers also completed a survey instrument 
which dealt with the relative importance of different 
types of policy relevant Information from educational 
studies. Staffers were asked to rank order the Importance 
of information and .knowledge needed in the policy process. 
Anonymity for the participants was assured. (63) 
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CODE 



08(09) 



DIMENSION DESCRIPTION 

Unit of Analysis........ ^ (po1lcymakers) 01 02 03 04 05 06(w)o8 09 

S "* l1n9 Convenience . ®° Z 03 04 07 08 09 

°* s1 9 n Not Codable 01 02 03 04 05 06 07 

Research Methods. Multiple Methods ' 01 02 03 04 05 06( 07 1)8 09 

Analytic Methods Rat1ngs . ,103 4 5 8 9 

Analytic Focus Non-qualitative . 1@8 9 

Reliability Not Sported 02 8 

Validity Not reported (l) 2 8 

Definition of Use [l)z 3 8 9 

Instrumental Vy 

Object of Use , 12.34568(9 

Uncodable 

Practice Area and 

Population Education, Congressional policymakers 
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EFFECTS OF COGNITIVE STYLE AND COUNSELOR-CLIENT 
COMPATIBILITY ON CLIENT GROWTH 

C 



P.S. fry^ and P. A. Charron 



Fry, P.S. and P.A^Ch^rron. "Effects of Cognitive Style 
and Counselor-Client Compatibility on Client Growth." 
journal of Counseling Psychology . Vol. 27, number 6(1980) 
529-538. ^\ 



The study was developed to Investigate the effects of counselor 
client cognitive style compatibility. 



Two cognitive style variables that had previously been shown 
to be very promising in a variety of interpersonal Inter- 
action studies—Serial 1sm-hol ism and field dependence-1n- 
dependence—were tested. Dependent variables Included the 
following: (a) client's scores on two ratings of Improvement 
in self-exploration skills and self-awareness; (b) counselors' 
and client's subjective evaluation of ease of relating 
with partners; (c) counselors' written counseling plans 
analyzed according to instrumental and counseling strategies. 



Field dependence-Independence was assessed by means of a 
Group Embedded Figures Test (GEFT) 1n which the subject 
must locate a previously seen simple geometric figure 
within a complex figure designed to embed 1t. The test 
is composed of twenty-four pairs of simple and complex 
figures. j 

Seriallsm-holism was assessed by means of a modified version of 
Gandlemullor Test wMch employs machine programmed systems for 
exhibiting learning strategies ana regulating uncertainty 
in a progranmed conversation between the learner and machine. 

The definition of cognitive style was adapted from the 
hypothetical construct developed by Goldstein and Blackman 
(1978) and Messlck (1976). The variables of cognitive style 
were adapted from studies by Bracht, 1970; Pask, 1975 and 
Saloman, 1972 (for Interpersonal interaction), and by 
Goldstein and Blackman, 1978, W1tk1n et ai, 1977 (for 
field dependence-independence). 



(Hi 



Witkin et al., (1962) reported conclusively that the GEFT 
has a high test-xetest reliability of :89 and .89 for men 
and women,ipespect1vely. 

The empirical finding reflecting the validity and reliability 
of the Gandlemuller Test was that a respondent with hoi 1st • 
and serialist disposition in the protest situation unequi- 
vocally adopts a holistic learning strategy in the programmed 
conversation. 



Cognitive style tests were Administered in groups of ten 
individuals to a broad sample of sixty clients and sixty 
counselor trainers. The §EfT was administered first and the 
Serial ism-holism test was/administered second. Thirty-two 
clients and thirty-two trainees were selected, then the 
subjects were categorized in terms of thirty- two client- 
counselor pairs that varied 1n terms of field-Independent 
versus field-dependent and serialist versus holist 
dimensions. Sixteen pairs represented extremes of scores 
and a high degree of matching or mismatching. The second 
set of ten pairs in the middle-range scores were rather" ' 
loosely matched or mismatched. The final analyses were based 
on a collapsed form of basic design.. 

Directly after the second session, counselor trainees and 
clients were seperated and asked to evaluate their partner's 
performance, with the help of a checklist. These checklists 
designed to obtain ratings on a four-point scale, were first 
administered immediately on the completion of the second 
counseling session. 



Bracht, G.H. "Experimental Factors Related to Aptitude- 
Treatment Interactions'.' Review of Educational Resea rch, 
1970, 40, 626-645. 

Pask, G. "Strategy, Competence and Conversation as Detrimenrs 
of Learning." Programmed Learning , 1969, 6, 250-267. 

Pask, G. Conservation , Cognitive and Learning : A 

Cybernetic Theory and Methodology . Amsterdam: Elsevier, 

Goldstein, K.M. and S. Blackman. Cognitive Style . New York: 
Wiley, 1978. 

WilJcin, H.A., C.A. Moore, D.R. Goodenough and P.W. Cox. 

T1 eld-dependent and Field- Independent Cognitive Styles and 
Their Educational Implications." Review of Educational 
Research , 1977, 47, 1-64. 



9 * * * 

52 



STUDY PROFILE 



DIMENSION DESCRIPTION CODE 



Unit of Analysis Dyad 0102)03 04 05 06 07 08 09 



Sampling Purposive 01VQ2J03 04 07 08 09 

Design co^ro'lT"* 1 St , atiStical 01 02 03 ^05 06 07 08 09 



Research Methods..; structural observation 01(02)03 04 05 06 07 08 09 

Analytic Methods Ratings 

103 4 5 8 9 

Analytic Focus Qualitative 

,©2 8 9 

Reliability.. Reported 1 © 8 



> 



Validity No estimate reported Q 2 8 

Definition of Use Conceptua1 1©3 8 9 ^ 

OMeetof Use -- Uncd.ble 1 2 3 4 5 680 

Pooulat1oJ rea Psychotherapy, counselors, counseling psychologists, 

p personality researchers 
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CHANGE SEEKER INDEX 



AUTHOR: 



Warren K. Garlington and Helen E. Shimota 



AVAILABILITY: 



Warren K. Garlington, Ph.D., Department of Psychology, 
Washington State University, Pullman, WA 99164. 



& PURPOSE.: 



f! 115?! J 5"? Shin?0t ? ^eorized that change-seeking need 
Is reflected in certain (unspecified) personality 
characteristics and hence should be measurable by an 
appropriate personality test. 



VARIABLES: 



Change-seeking need (perceived desire for activity 
novelty, or excitement in lifestyle). 



DESCRIPTION: 



This is a ninety-five item, forced choice (true-false), 
self report inventory. Some sample items of the Change 
Seeker Index (CSI) are: 

1) I think a strong will power is a more valuable gift 
than well-informed violence. 

2) I like to read newspaper accounts of milrders and other 
forms of violence. 

V 

3) I like to conform to custom and to avoid doing things 
that people I respect might consider unconventional. 

4) "I would like to see a bullfight in Spain. 



DEVELOPMENT: 



A preliminary item pool was constructed by selecting 
items on an unspecified basis from existing personality 
inventories that seemed to reflect change seeking. The 
authors constructed the additional items! The preliminary 
inventory was administered to a heterogenous Sample of 
105 college^men, 137 college women, and sixty male soldiers. 
nnal selection of items was based on item/total score 
correlations. Sepearte item analysis were performed 
within each of the three sample groups. The ninety-five 
items that correlated significantly (p<.05 in at least 
^two of the three groups were retained. 
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RELIABILITY/ 
VALIDITY: 



Split-half coefficients of .85 (n=80) and .80 (n=50) 
were reported for two undefined groups of college students 1 . 
A test-retest coefficient qf .91 (n=48) was reported for a' 
group of hospitalized psychiatric patients tested 1 to 3 
days after admission and again 7 to ten days later. A 
test-retest coefficient ;od .77 (n=44) was reported for a 
group of soldiers tested after an interval of three minutes. 

Some evidence of construct validity was cited. Intelligence 
Quotient and CSI scores did not correlate significantly 

(r= "; 1 f , J n= ? 3) for a qrou P of Psychiatric patients. Age 
correlated slightly and negatively (V* -.21, n=150, p <.01) 
for a sample of 150 patients. Men also scored higher than 
women. These relationships were claimed to be expected by 
the conceptualization of change seeking. 



ADMINISTRATION: 



Self-administered; appropriate length of time for completion 
is not specified, but forty-five minutes would appear to be 
ample for most respondents. No information on scoring is 
provided, but it appears that an item marked "true" could 
be assigned a value of 1, an item marked "false" could be 
assigned zero (0), reverse scored items could be assigned 
1 if answered "false", and the respondent's score would be 
a total of assigned values. 



SOURCES: 



Garlington, W.K., Helen E. Shimota. "The Change Seeker 
Index: A Measure of the Need for Variable Stimulus 
' Input." Psychological Reports , Vol. 14, (1964) : 919-924. 

Klahn, J. E. An Analysis of Selected Factors and Surrey 
of First Year Student Nurses . Unpublished doctoral 
dissertation, Washington State University, 1966. 

Klahn, J. E. "Self-concept and Change-Seeking Need of 
First, Year Nurses." Journal of Nursing Education, 
.(April 1969):11-16. * U 
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STUDY PROFILE 

DIMENSION , DESCRIPTION CODe( 

; ' 3 

■ / 

Unit of Analysis _ , _ (01J02 03 04 05 tnp 06 09 

Individual V-x 

Sampling 0 . 01(62)03 04 07 08 09 

Purposive v \-s^ 

' 

***** — Uncodable ' 01 02 03 04 05 06 07 08@ 

Research Methods........ QMestionnaire 01 02 03 04 05@07 08 09 

Analytic Methods Bivari ate Correlation Analysis 12 Q A 5 8 9 

Analytic Focus Qualitative O 2 8 9 

Rel iabi 11 ty : . . . . Reported 1 (2) 8 

Validity ..Reported 1©8" 

_ Instrumental with change /T"\- „ 

Definition of Use in behavior {^J 2 3 8 9 

Object of Use Scientific study 1 © 3 4 5 6 8 9 

Practice Area and 

Population Psychiatry, Nursing, general population 
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THE CARAVAN ROLLS ON 



Gary Gregg, Thomas Preston, Alison Geist and Nathan Cap! an 



Gregg, Gary, Thomas Preston, Alison Geist and Nathan Caplan. 
"The Caravan Rolls On: Forty Years of Social Problem 
Research", Knowledge : Creation , Diffusion , Utilization , 
Vol. 1, number 1, (September 1979):31-61. ^ 



Gregg et al . undertook an examination of the "way social 
science studies social problems," Research reports were 
collected and analyzed in order to assess their relative 
qualities m terms 6t topic, context, and method. 

Variables measured were: type of journal, type of article 
(research, program evaluation or report, literature review), 
independent variable measured (ranging from personal char- 
acteristics to system characteristics), causal attributions 
made, relevance to theory or practice, and type -of theory 
or practice under consideration. 



Samples of social science literature were taken from the 
years 1936, 1956, and 1976, Six hundred and ninetyreight 
articles were examined in six "social problem" areas: 
alcohol and drug abuse, suicide, delinquency, job satisfaction 
rape, and race relations. Source journals were both, pure * 
and applied. 



3,354 independent variables were coded and grouped into 
seventeen categories according to patterns pf causal 
attributions reported by the authors of research articles. 



The project was undertaken in order to explore more fully 
the findings of Caplan and Nelson (1973) which indicate 
that the institutional framework within which social 
science occurs favors research in personal factors and 
discourages research in social or organizational factors. 
The theoretical background to the research lies in the belief, 
expressed by numerous analysts, that the sciences (natural 
and social) "do not progress autonomously in accordance 
with their own rigorous rules....", but should rather be 
explained by "'external 1 factors ranging from generational 
conflict within the scientific community to the investment 
of competing theories with symbolic significance in major 
political struggles" (Kuhn, 1970, 1977, Feyerabend (1975) 
and Merton (1973). (See "Sources" below.) 



RELIABILITY/ 
VALIDITY: 
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No estimates on reliability or validity were reported. 



ADMINISTRATION: 



Each of the 698 'articles was coded according to the class- * 
ification schema outlines above. The major findings concerned 
the percentage of articles which included attributional 
patterns (only 7% involved "multiple complex" relationships), 
the focus of study (the percentages of person-variables 
and system variables studied remained relatively constant 
over time- about 60X person and 20% system,) *nd the, 
relationship between social forms and research topics. 
There were significant instances of research variables and 
theoretical approach varying over time in seeming response 
to social movements (e.g., the study of racism and 
discrimination). The penultimate conclusion, based on the 
completed analysis, was that the most serious failure of 
"social science sua science [is] its preoccupation with 
breaking social pHenomena sown into collectives of facts 
that cannot be reintegrated into a model of society." 



SOURCES : Caplan, N., and S.D. Nelson. "On Being Useful: The Nature 

^— and Consequences of Psychological Research on Social 

Problems." American Psychologist . Vol. 28 (1973) : 199-2. 

Feyerabend, Paul. Against Method : Outline of An Anarchistic 
Theory of Knowledge . New York-: Schocken Books, 1975. 



Kuhn, Thomas. The Essential Tension : 
in Scientific Tradition and Change , 
of Chicago Press, 1977. 



Selected Studies 
Chicago: Unr^rsity 



The Structure of Scientific Resolutions , Chicagp: 

University of Chicago Press, 1970. 



Merton, R.K. The Sociology of Science : 
Empirical Investigations . Chicago: 
Press, 1973. 



Theoretical and 
University of Chicago 
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STUDY PROFILE J 

DIMENSION DESCRIPTION CODE 

Unit of Analysis Individual ©02 03 04 05 06 07 08 09 

Sampling Purposive 01©'S 04 07 08 09 

Lpngltudinal or crossectional . 

Design without statistical controls 01 02 03\04)05 06 07 08 09 

Research Methods.. Content Analysis 01 02@ 04 05 06 07 08 09 

Analytic Methods Blvariate Analysis 1 2 (j) 4 5 8 9 

Analytic Focus Non-qualitative i 1 (?) 8 

Reliability (l) 2 8 

Not reported ^ 

Validity.... Mot reported © 2 8 

Definition of Use „ . 1 Cl) 3 8 9 

Conceptual ^ 

Object of Use. stu<jy 10345689 

Practice Area and 

Population Scientific communities, authors 
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INTERPERSONAL TRUST DIFFERENTIAL 



AUTHOR: 



Kim Griffin 



AVAILABILITY: 



Griffin, Kim. "An Experimental Evaluation of the Trust 
Differential." Lawrence, Kansas: The Communication Re- 
search Center, University of Kansas at Lawrence, 1968. 



PURPOSE : The Interpersonal Trust Differential is designed to measure 
the level of interpersonal trust within defined groups. 



VARIABLES : Twenty-seven semantic-differential Items measure aspects 

of an individual^ perceptions about other members of 

.a group or the group as an entity. 
Sample items are: 



(1) Scholarly _ 
(JLO)Bond 
(lft) Introverted* 



Unscholarly 
Timid 

Extroverted 



DESCRIPTION : ' The twenty-seven semantic differential items answered in 

a, seven point Likert-type scale. Subsequent Factor Analysis 

and Calculation of T-ratios yielded three factors: 
character, dynamism, and expertness. A total of 325 
respondents, divided into fifty-three small groups, took 
part in the revision of an earlier instrument which 
formed the basis of this study. 
t t 

DEVELOPMENT : The current questionnaire was developed from an earlier „ y 

seventy-two semantic differential scale. Each of the 

original items was analyzed in order for reliability 

and validity. (No details on the results of this analysis 

were presented.) < 



RELIABILITY/ 
VALIDITY: 



No figures for reliability or validity were given for the 
revised twenty-seven item survey. 



ADMINISTRATION : 



The items are presented on a self-administered question- 
naire. Average completion time is between ten and twenty 
minutes. 
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MEASURING STAGES OF CONCERN ABOUT THE INNOVATION 



Gene Hall, Archie George, and William Rutherford 

Hall, Gene, Archie George and William Rutherford. 
Measuring Stages of Concern About The Innovation ; A 
Manual for Use of~ the SOC Quest 1 on naTrtT Austin: ~ 
ResearclTandDevelopinent Center for Teacher Education, 
University of Texas at Austin, 1979. 



The SOC questionnaire is designed to assess the concerns 
of both users and nonusers concerning implementation of 
a specific innovation. The questionnaire is intended to 
provide a relatively quick and easy way for manager or con- 
sultants to identify concerns of his/her clients. 



The SOC questionnaire hypotheses seven stages of concern 
which an individual confronting a recently introduced 
innovation will take on to varying degrees. The stages are: 
1) Awareness (little concern or involvement with the in- 
novation; 2) Informational (a general awareness of the 
innovation and an interest in its substantive aspects; 3) 
Personal (an uncertainty exists about the relationship 
of the individual to the innovation), 4) Management (focusing 
attention on the introduction of the innovation on a daily 
basis),' 5) Consequence (focusing attention on, the impact 
of the innovation on subordinates and co-workers), 6) 
collaboration (focusing on cooperation and coordination with 
co-workers regarding the Innovation), and 7) Refocuslng/ 
(the exploration for new innovations as replacements. 



The SOC questionnaire consists of thirty-five items, to which 
individuals respond. The respondent marks each item on a 
0 to 7 Likert-type seale according to the perceived truth 
of the item to the respondent. 

The Research and Development Center for Teacher Education 
at the University of Texas at Austin developed this pro- 
cedure out of a desire to more fully explore the effects 
of innovations on individuals. The process of change often 
brings with it a number of complex feelings of uncertainty, 
doubt and insecurity rooted in an individual!* desire to 
come to grips with innovations. Other procedures centering 
around this concern examine Innovations as a change process 
focusing on individuals' feelings about the innovation / 
("Assessing open-ended statements of Concern About an In- 
novation"— See pagell8). The rate and way in which an 
innovation is used ("Level of Use"— See page ), and 
the variations of the innovation which are arrived at by 
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individuals as the Innovation is disseminated ("Innovation 
Configuration"--See page ). . 



RELIABILITY/ 
VALIDITY: 



The alpha coefficients of internal consistency for each 
of the seven stages are: Awareness: .64, Informational: .78, 
Personal: .83, Management: .75, Consequence: .76, Collabor- 
ation: .82, and Refocusing: .71. (n=830). Test-retest 
correlations for each of the stages are: Awareness: .65, 
Informational: .86, Personal: .82, Management: .87, 
Consequence: .76, Collaboration: .84, and Refocusing: .71. 
(n«171). 

Validity was measured by subjecting 150 of the 195 total 
items to principal components factor analysis with varimax 
rotation. Correlations between the seven identified factors 
and the seven stages of concern were: 

Correlations Between Varimax Factor Scores- 
and SOC Scale Scores 



SOC 

Scale 

Scores 



0 
1 
2 
3 
4 
5 
6 



Varimax Factor Scores 
1 6 3 4 2 5 



.83 



.67 



.72 



.91 



.96 



.82 



.88 



FROM: Measuring Stages of Concern About the Innovation : A Manual 
pr Use of the SOC que stionnaire . Austin: Research and 
Development Center for Teacher Education, University of Texas 
at Austin, 1979. 



ADMINISTRATION: 



The SOC Questionnaire is self-administered and requires 
between ten and fifteen minutes to complete. Scoring can 
be done with a computer or hand scored. The raw score 
for each of the seven scales is simply the. sum of the 
Likert-type responses to the five statements for that scale. 
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Hall, Gene, Susan Loucks, Will 1am Rutherford and Beulah 
Newtone. "Levels of Use- of the Innovation: . A 
Framework for Analyzing Innovation Adoption." Journal 
of Teacher Education . Vol. 26., number 1, (1975) :52-56. 
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DIMENSION 



STUDY PROFILE 



DESCRIPTION 




CODE 



z : 1 

Unit of Analysis Individual 

Sampling , 

Design 

Research Methods i 

Questionnaires 



Purposive 



Quasi -Experimental without 
control groups and/or - 
r a n do mi sa ti o n 



Analytic Methods Ratings/Codes 



Analytic Focus.. Non-Qual1t1t1ve 

f 

Reliability .... 



Reported 



Validity..;. 



Reported 



Definition of Use....... instrumental 

Object of Use ; Study 



(oT)o2 0$ 04 05 06 07 08 09 



0UQT ( 03 04 07 08 09 



02 03 04 05 06 07 08 09 10 



01 02 03 04 05r06 07 08 09 



1 (2 ?3 4 5 8 9 



1 (2 s , 8 9 
0-2 8 

02 8 
(T}2 3 8 9 
1(2) 3 4 5 6 8 9 



Practice Area and 

Population General, Education, Practitioners 



/ 
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JOB SATISFACTION AND THE DESIRE FOR CHANGE 



E1nar Hardin. 



Hardin, Einar. "Job Satisfaction and the Desire for 
Change". Journal of Applied Psychology , Vol 51, number 
1, 1967:20^277"" *~ 



To test the hypothesis that a person's desire for specific 
changes 1s governed by the discrepancy between the 
attractiveness to him of existing and potential job 
characteristics, and by an assessment of the process of 
change or passive general readiness for change. 



Variables measured are Job Satisfaction, Desire for Change, 
and Passive Readiness for change. 



The survey constituted a questionnaire of thirty-seven 
Items dealing with desire for change, job satisfaction, 
and readiness for change. For the desire for change 
variable, the employees were asked to indicate whether 
they were "completely satisfied" or "hot satisfied" with 
each aspect. They were also asked to Indicate if they 
wanted to see "an Increase", "no change", or " a decrease" 
in that aspect. 

#he level of satisfaction with an aspect was expressed 
numerically by assigning the arbitrary weights of 
1, 2, 3, 4, and 5 to the five categories of response 
from "completely satisfied" to "not satisfied". 
Readiness for change was measured by response stating - 
that "I strongly agree", "I agree a little", and "I 
neither agree nor disagree", "I disagree a little", and 
"I strongly disagree". The statements were as follows: 

1. If I could do as I pleased, I would change the kind 
of work I do every few months . 

^2. One can never feel at ease on a job where the^ways 
of doing things are always being changed. 

3. The amount of variety in my work. 

4. My control over the pace of my work. 



J 



5. The importance of my job for the company. 



The material for this study was obtained from the flrsjt - 
of two questionnaire surveys that had been conducted by the 
author and associates 1n an earlier longitudinal study 
of employee response to technological change. The 
questionnaires completed by^246 employees who participated 
in both surveys constituted tW basic data pool 7] 



The Product-Moment Correlations among test items had a 
median of .20 for Readiness of change, .51 for job 
satisfaction, and .26 for Desire for change, (n=»199). 
The reliability of the two fourteen-item scores appeared 
to have been fairly high. However, the readiness-for- 
change items appeared fairly heterogenous. 



The items were presented in order, but not in the form 
of a checklist. The ninth item was of the controlled- 
completion type, that 1sTT the job that you would consider 
ideal for you would be one where the way you do your work: 
is always the same; changes very little; changes somewhat; 
changes quite a bit; changes a great deal. A total score 
on readiness for change was computed by assigning values 
from 1 to 5 to the response categories ranked in ascending 
order of readiness and summating the weights. 



Lippitt, R, J. Watson, and B. Westley. The Dynamics 
of Planned Change : A Comparative Study of Principles 
and Technfques . New York: Harcourt, Brace, 1958. 

Trumbo, D.A. "Individual and Group Correlates of-Attitudes 
Toward Work-Related' Change? Journal of Applied 
Psychology , Vol. 45" (196l£a33-34T 

Hardin, E. "The Reactions of Employees to Office 
Automation 1 '. Monthly Labor Review , Vol. 83 (I960): 
925-932. 

Zander, A. "Resistance to Change-It's Analysis and Prevention 
Advanced Management , Vol. 15 (1950) :9-ll. 
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STUDY PROFlbE ' 



DIMENSION DESCRIPTION £ CODE 

Unit of Analysis . 4 01 02 03 04 05 06 07 08*09 

Individual % 

Sampling. ; D ..™*„- 01 02 °3 04 07 08 09 

* Purposive 

Quasi -Experimental without 

Design control group and/or 01 02 03 04 05 06 07 08 09 

randomization / 

Research Methods Questionnaire 01 02 03 04 05 06 07 08 09 

Analytic Methods Multivariate analysis 1 2 3 4 5 8.9 

Analytic Focus Non-qual1tat1ve 1 1 2 i* 9 

Reliability Reported 12 8 



Vanity Not-reported 1 2 8 

Definition of Use v Conceptual. 1 2 3 8 9 



Object of Use Management innovation 1 2 3 4 5 6 8 9 

X 

Practice Area and 

Population Psychology, Management employees * & 
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MEASURING INNOVATION CONFIGURATIONS- ) \y^~ 
PROCEDURES AND APPLICATIONS 



Susan Heck, Suzanne Stiegelbauer, Gene Hall and Susan 
Loucks. a 

Heck, Susan, Suzanne Stiegelbauer, Gene Hall and Susan 
Loucks . Measuri ng Innovation Configurations : Proced ures, 
ajd Applications. Austin, .Texas: Research an d Develop- 
ment Center for Teacher Education, University of Texas, 1981. 

Innovation Configurations represent operational ized patterns 
of Innovation characteristics which result fr^m implementa- 
tion by individuals 1n specific contexts. Different In- 
dividuals will use segments of a parttcular Innovation in 
different ways. The /systematic explication and assessment 
of these patterns forms the basis for the Innovation 
Configuration Checklist. 

* • * 

Variables represented are the behavioral ly observed patterns 
of Innovation use which represent the checklist. The 
stages pf concern about an innovation (See Procedure number 

k * pa 5 e ^ L? nd levels of use of an Innovation may also 
be used with this procedure. Taken together, these three- 
Innovation Configuration, Stages of Concern, and Level of 
Use-may be- used as "key diagnostic variables that a change 
facilitator must monitor" 1n order to" affect outcomes of 
Innovation, adoption and implementation. 

The assessment of Innovation Configurations was intended 
by the author's to be used in research,' evaluation, staff 
development and dissemination studies. While the scope, 
amount of detail, specific format, and type of material' 
Included 1n any particular checklist will vary with the 
research context and the type of information desired by 
the researcher, the basic format for a checklist is a series- ' 
of statements and questions respondents must answer •concern- 
ing the innovation. The researcher, after outlining the 
specific Innovation to be studied, identifies the components- 
iT^°Il! eatyre f" of the innovation, usually through extensive 
•J t ? 1 W re reviews and interviews with key actors and 
stakeholders. 

Thexheckl 1st containing component parts of the innovation 
is divided into those which are "ideal", "acceptable", 
and "unacceptable" according to the mod<?l of the innovation 
which the researcher has arrived at. For details on 1 the 
administration, and analysis of checklists, see "Adminstration. 
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The Innovation Configuration checklist is part of the 
N larger research on innovation adaption and implementation 
bejng done at the Research and^evelopment Center for 
Teacher Education at the University of Texas at Austin. 
The Theoretical Mode of which the Checklist is a part - 
views change "as a process, not as an event", and examines > 
innovations as a change process focusing on individuals^ 
feelings about the innovation (Stages of Concern Scale- 
See page ), the rate and way in which the innovation 
is used, (Level of Use-See pag* ), and the variations 
of the innovation whi'ch are arrived at by various in- 
dividuals as the innovation is disseminated {Innovation 
Configuration). 



Reliability and Validity have not been calculated. 

*) * 

The Innovation Configuration Checklist (See "Development") 
is given to respondents involved in innovation use. 
Self -administered questionnaires, interview, or observation 
may be u$ed in administering the Checklist. Length of 
of these typed of administration varies with the 
8nt of detail desired and. type* of . Innovation under 
Tsi deration. 

Data Analysis normally consists of frequencies-how many 
times Individuals use each component of the innovation , 
or a variation. Clustering of components may be done by 
hand, and yield "dominant" configuration patterns. Re- 
lationships between configuration patterns and outcomes 
may also be gathered— for example, the extent to which 
certain configuration patterns correlate with outcomes 
such as achievement test scores, profpotions, etc. v 

Heck, Susan, Su^Jfme Stiegelbauer; Gene Hall and Susan 
Loucks. Measuring Innovation Configurations . Austin: 
Research and Development Center for Teacher. Education, 
University of Texas at Austin, 1981. 

Hall, Gene and Susan Loucks. % "A Developmental Mod?! for 
Determining Whether the Treatment Is Actually Implemented 
American Educational Research Journal , Vol. 14, number 3, 
(Summer, 1977):263-76. 
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STUDY PROFILE 

DIMENSION DESCRIPTION CODE 

» ' . . . 

Unit of Analysis........ Individu?1 Pj^02 03 04 05 06 07 08 09 

Sampling Purposive , 01 02 03 04 07 08 09 

Quasi -Experimental without 

Design Statistical Controls 01 02 03 04 05 06 07 08 09 

Research Methods 01 02 03 04 05 06. 07 08 09 

Multiple Methods J < 

Analytic Methods B 1vari ate Analysis 1 2 3 4 5 «. 9 

. , ^* r- Qualitative , „ „ . 

Analytic Focus '^2 8 9 

> ■ 

Reliability i •* Not Reported 1(2,8 



Validity.. Not Reported 12 8 

Definition of Use. Instrumental 1 2 3 8 9 

* » 

Object of Use Policy/Management 1 2 3 4 5 6 8 9 

Innovation * 



V 



.Practice Area and 
population. Education, Practitioners, Policymakersj 



9 
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HILL INTERACTION MATRIX 



AUTHOR: 



William Fawcett Hill 



AVAILABILITY: 



Hill, W.F. Youth Studies Center, University of Southern 
California, Los Angeles, CA 90007. 



.PURPOSE: 



Hill Interaction Matrix was- developed to serve as a systematic 
conceptual framework for understanding, describing and doing 
research on group psychotherapy. 



VARIABLES : Verbal interaction is classified along two dimensions found " 
, useful in distinguishing yarious, types of groups. One dimen- 
sion focuses on the content of group activity (what the group 
talks about). Four categories are used for this dimension: 
t I) Topic , topics external to actual group concerns; II) Group , 
conversation about group itself; III) Personal , about problem " 
of a group member; IV) Relationship , talking about the "here- 
and-now" relationships of members to each other. 

Five categories comprise the second dimension of the matrix, 
work style : A) Responsive; B) Conventional; q) Assertive; 
D) Speculative; E) Confrontive. 



* 



DESCRIPTION: The HIM is not an instrument, but a scoring system. Each ver- 

; bal interaction is assfgned to one of the 20 cells of the 

matrix, which is four-by-five, corresponding to the four 
content categories and the five work style categories. 

DEVELOPMENT : The first stej) 1n the development of HIM was empirical. Sev- 
eral hundred psychotherapy group meetings were observed, re- 
corded, transcribed, and studied intensively. Judgments of 
observers were compared with comments elicited from the group 
participants, during stimulated recall sessions. From this 
"study a two-dimensional matrix was developed into the current 
20-cell form. « 

The second step involved interviewing a large number of group 
psychotherapists, to determine essential theoretical ingredi- 
ents in group psychotherapy. The empirical categories 'were 
then compared with psychodynamic theories, and placed on a 
hitrarchy of therapeutic significance,. \As the scales were^ 
tested against further groups, revisions! were made in the 
original matrix (HIM '54, HIM '56, HIM ^62). 
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RELIABILITY/ 
VALIDITY: 



; '1 

Some reliability data have been collected on statement-by- 
stateraent forms of HIM. Using three trained judges on three 
groups, interjudge agreement averaged 70%. Using an unambig- 
uous "typical statement" -64-itera card sort, judges' reliability 
Was 90%. The newer 72-item rating scale, HIM-62, has not been 
completely checked for reliability. 

Validity claims are based on HIM's demonstrated ability to 
distinguish between therapy groups representative of several 
schools of psychotherapy. 



ADMINISTRATION; 



The HIM is filled out by a judge, observer, group leader, or 
other scale user after listening to a tape, reading a session 
transcript, or viewing a group session. The rater indicates 
for each item how frequently the behavior in question appeared 
or how many members participated. For each cell above the A 
level of group activity there are four items, two describing 
member activity and two describing reader activity. In addi- 
tion, there ts one item for each of the A level cells and four 
non-specific items. The ratings are entered on IBM cards 
and stored by computer. The computer result tells the inter- 
action occurring in each dimension of the total group pattern. 



SOURCES: 



1. 

2. 
3. 



Hill, W.F. Hill Interaction Matrix 

Interaction in PsychotHerapy Groups . Los A"ngeles! C, 
Youth Studies Center, University of Southern Califor 
Hia, 1965. 



A Method of Studying 



Locke, N. Group Psychoanalysis . 
University Press, 1961. 



New York, NY: New York 



McCarthy, R.A.' "Group Therapy in Alcoholism." Quarterly 
Journal of Studies of Alcohol , Vol . 11 (1950): 309. 
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STUDY PROFILE 

DIMENSION DESCRIPTION CODE 

s • ■ 

Unit of Analysis... Group 0102^)04 05 06 07 08 09 

Sampling.. Purposive 01 <65)03 04 07 08 09 

. / 

Descriptive case study < ^- > 
Design with explicit theory 01(02)03 04 05 06 07 08 09 * 

Research Methods Content analysis 01 02© 04 05 06 07 08 09 * 

Analytic Methods Ratings 1 © 3 4 5 8 9 

Analytic Focus Qualitative i (5 2 8 9 

Reliability.; Not reported ©28 

Validity : Not reported t CD 2 8 

: * 

Definition of Use Conceptual 1 © 3 8 9 * 

; 1 1 1 ■ 

Object of Use. ^Other 1 2 3 45(5)8 9 

Practice Area and 

Population........; Group Psychoanalysis and Psychotherapy, 

Mental patients* 
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THE EDUCATIONAL INFORMATION MARKET STUDY 



73 



Paul Hood and Linda Blackwell 

Hood, Paul ajnd Linda Blackwell. The Educational Information 
Market 'Study^ San Francisco, CA: Far West Laboratory for 
Educational, Research and Development, October, 1976. 

This study was concerned with defining the characteristics, 
needs, and purposes of educational audiences in terms of 
their actual or potential use of educational Information. 

This study 1s based on an Educational Information Use Model 
1n which the authors establ 1 shed relationships among several 
sets of variables Including: (Ijf organizational context; 

(2) position and kind of work performed by the respondent; 

(3) purpose for seeking Information; (4) availability of 
Information sources; (5) Information channels used; (6) 
Information used/preferred; and (7) demographic variables. 



Six versions of a" seven-page Instrument were created 1n 
order to tailor questions concerning (a) needs for Information 
1n broad subject areas, (b) work activities, and (c) people 
and organizations users turn to 1n seeking advice or Infor- 
mation, to the special characteristics of (1) practitioners, 
(2Va<ta1nfstrators, (3) higher education chiefs and Institu- 
tional researchers,- (4; educational faculty and social scien- 
tists, (5)^school boards, and (6) legislators. 

The questionnaire was organized 1n nine major sections dealing 
with questions: (1) about yourself and -your work, ^(2) about 
the Information sources you use 1n your most important work 
activities, (3) about the usefulness of the information 
sources you use, (4) about the most important characteristics 
of the Information sources vou prefer, (5) about your purposes 
for seeking Information, (6) about your problems in acquiring 
and using Information, (7) about the people and organizations 
you turn to, (8) about the Information products and services 
that would be most useful^to you, and (9) statistical data 
(age and degree attained) (1-2). 



The Educational Information Market Study was part of a larger 
itudy of educational information system requirements, spon- 
sored by the National Institute of Education. The study was 
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a two stage effort involving a field interview and a mail 
questionnaire. Field Interviews that were % conducted with 
a judgmentally-s elected sample of 137 key. persons, repre- 
senting 17 different educational roles, and located in 
over forty communities throughout the United States. The 
field survey was undertaken to develop an Indepth under- 
standing of user information needs, to develop and refine 
a conceptual framework and associated data-analytic methodo- 
logy, and to provide the basis for the design of a compre- 
hensive, hatloppide probability sampling mail survey of 
all major types of users. 



None reported. 



Four major audiences (and fourteen subaudiences) were iden- 
tified: (1) elementary and secondary level public school 
practitioners (teachers, principals, other Instructional and 
support staff), (2) elementary and secondary public education 
adninistrators and professional staff (local, Intermediate, 
and state agencies), (3) higher -education groups (chief 
adnlnlstrators, Institutional researchers, faculty of schools 
and colleges of education, social scientists), and (4) 
governance groups (local school boards, state school boards, 
state legislators, U.S. Congress1oi|n aides). An overall 
response rate of approximately fifty percent was achieved; 
however, response rates for subaudiences ranged from tWenty- 
three percent for state legislators to sixty-nine percent 
for higher education chief administrators. 
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STUDY PROFILE 

DIMENSION DESCRIPTION CODE 

" ft I 

Unit of Analysis Other (educational audiences) 01 02 03 04 05 06@)08 09 

SafpUng R andora 01 02@04 07 08 09 

Design.... Not codable 01 02 03 04 05 06,07 08 @ 

Research Methods Multiple methods 01 02 03 04 05 06 ©08 09 

Analytic Methods Multivariate 1 2 3 (5) 5 8 9 

Analytic Focus Nonqual itative ' i 1 @ 8 9 

Reliability Not reported Q) 2 8 

Validity Not reported ® 2 8 

Definition of Use Instrumental 0 2 3 8 9 

Object of Use Other 12 3 4 5 (f) 8 9 

Practice Area and 

Population Education practitioners and policymakers O 
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DISCOVERING STRATEGIC PATTERNS FROM DOCUMENTARY EVIDENCE 



f) 

AUTHOR: ' 



Anne Slgismund Huff 



AVAILABILITY: 



Anne Siglsmund Huff, Department of Business Administration, 
University of Illinois at Urbana-Champaign. 



PURPOSE : This study has as its primary purpose the discovery and 

systematic analysis of strategic patterns from documentary 

analysis. 

VARIABLES : Variables in the study were the causal attribution statements 
. njAde in documentary evidence-speeches of a school super- 

intendent and the patterns these attributions fell into. 

DESCRIPTION : Severtty-three speeches made bv one school superintendent 
' over fifteen years (1965-1980) were acquired and analyzed. 

Causal statements were identified and coded according to 

the following coding schema: 

CODING CLASSIFICATION SYSTEM 



Symbol 



Definition 



1+i 
1-1 
1+1 
lol 

lal 
lml 
lol 

1-1 
1E1 



Positively affects 
Negatively affects 

Will not hurt, does not prevent, is not harmful 

Will not help, does not promote, is of no benefit 
to 

May or may not be related to, affects Indeterminacy 

Affects in some non-zero way 

Does not matter for, has no affect on, has no 
relation to 

Is equivalent to, is defined as 
Is an example of, is one member of 



DEVELOPMENT : A "strategy" -the focus of this research, is defined as 

"a pattern of key decisions over time " which do not 

necessarily have an explicit statement of intent to be 
discovered and analyzed. The method utilized in this 
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study which focuses on causal assertions, was developed by 
Robert Axel rod and his associates. 



RELIABILITY/ 
VALIDITY: 



Intercode reliability was estimated at 96% by Axel rod 
(1976:229-230; see "Sources" below). Intercoder reliability 
on the issue of whether or not statements were "causal" 
was estimated at 80% in the same source. 



No estimates of validity were given. 



\ 



ADMINISTRATION: 



Using the coding schema presented above, the researcher 
analyzed documents and presented a series of findings 
which Identified changes in "strategic patterns" of 
cause and intent by the superintendent. 



SOURCES: 



Axel rod, Robert (ed.). Structure of Decision . 
University Press, Princeton,- 1976. 



Princeton 
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STUDY PROFILE 



i 



DIMENSION DESCRIPTION CODE 

k - 



Unit of Ajp^TCls Other (documents) 01 02 03 04 05 06(07)08 09 



Sampling Purposive 01(02)03 04 07 08 09 



i Descriptive case study with ^-v 
Design explicit theory 01\£2)03 04 05 06 Q7 08 09 



Research Methods Content Analysis 01 02 (03/04 05 06 07 08 09 

Analytic Methods Ratings and codes 3 4 5 8 9 



Analytic Focus Non-qualitative , 108 9. 

Reliability Reported 10 



Validity 1 (7) 

Reported V_/ 



8 



8 



Definition of Use 1 (7\ 3 8 9 

Conceptual V^/ 



Object of Use c ^ A 1 [Z )Z 4 5 6 8 9 

Study 

Practice Area and rj rt ( \ 

Population Education, Policymakers 



INTEGRATING ACADEMIA AND PRACTICE 



Knowlton Johnson 



Johnson, K. "Stimulating Evaluation Use By Integrating 
Academiaand Practice," Know! edge : Creation , Diffusion , 
Utilization . Vol. 2, number 2 (December 1980):237-62. 



The* purpose of this study 1s to examine how to integrate 
the worlds of acidemia and practice so that evaluation ' 
research results will be more frequently used by decision- 
makers. The study focused on evaluation research products 
produced by academics for decisionmakers 1n a network of 
state and county organizations concerned with the admin- 
istration of just1ce(238). ' 

The dependent measure, evaluation utilization, was 
developed through a Guttman Scalogram Analysis of questions 
pertaining to conceptual use (thinking about or planning 
to use the evaluation method or results) and Instrumental 
use (using or suggesting to supervisor the use of * 
evaluation products). 



Conceptual use .of the evaluation products was operational ized 
in two ways. One measure operational ized the concept 
of program evaluability by combining two questions probing 
the respondents 1 potential use of the evaluation procedures 
(process evaluation) and the findings. A second measure 
concerned the extent to which decision makers "had developed 
their own Ideas fusing the findings or ideas of how process 
evaluation could be used to evaluate other programs in 
their agency. Respondents who indicated they had made 
specific plans to use the evaluation results or method 
emplo^d to produce the results were Included 1n the 
composite sicore of the second type of measure. 

Measures for four types of Instrumental use were constructed. 
First, a measure of general use was developed requiring no 
use of specific findings. This measure included questions 
about using findings for funding purposes and for justifying 
current direction. Second, the use of particular findings 
to modify a program was Included and, third, specific, use 
of findings for proqram development purposes was considered. 
A fourth measure of instrumental use concerned whether 
the respondents had presented or suggested that a formative 
type of evaluation procedure be used to evaluate other 
programs. 
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DEVELOPMENT: 



The conceptual framework of Havelock (1969) provided the 
starting point for the construction of an Academla-Practice 
Linkage Model. This framework relates resource-user 
linkage to four change process elements -who delivers what to 
whom by what channel . 



RELIABILITY/ 
VALIDITY: 



By using the described measures of evaluation use, a 
Guttman Scalogram Analysis revealed that a scale ranging 
from 0 to 6 could be constructed with a coefficient of 
reproducibility of 0.94 and a coefficient of scalability 
of 0.73. 



ADMINISTRATION: 



\ 

SOURCES: 



Interviews were conducted with seventy-five decisionmakers 
working in twenty-five organizations. Four trained graduate 
students conducted personal interviews with these agency 
personnel. The interviews took approximately 30-45 minutes 
to complete. 

The interview schedule contained a series of questions about 
the frequency and nature of contact with university personnel 
and students, exposure to the content of the evaluation 
products, and use of the evaluation findings. A question- 
naire solicited background information, appraisals of the 
evaluation producers, and their products and attitudes toward 
using such products. On completing the interview, the 
questionnaire was left for decision makers to complete 
and return to a governmental official coordinating the 
data collection stage of the study. A total of 68(91X) 
of the respondents returned the questionnaire. (244-245). 



Havelock, R.G. Planning for Innovation Through Dissemination 
and Utilization of KnowTedge . Ann ArEor: University of 
fflchigan, Institute for So<ii*i Research, 1969. 
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STUDY PROFILE 

DIMENSION ' DESCRIPTION CODE 

r~* — — — — — : — — — — 

Un1t ° f AnalySlS O^jgedslon makers) 01 02 03 04 05 «<© « 09 

Sampling Convenience , 01 02 93 04 07 08 09 



Design uncodabie 01 Q2 Q3 Q4 Q5 Q6 Q7 ^ Q ^ > 



Research Methods Mu1tjp1e 01 02 03. 04 05 06 07 08 

Analytic Methods........ ^IvaHate Analysis 1 2 3 4 5 8 9 

Analytic Focus.... Non-qualitative . 1 >2 8 9 / 

/ 

Reliability.... Reporte<j * 1 (z a__ / 

Validity ••••• Not reported 1 2 . 8 

Definition of Use Uncodabie 1 2 3 8 9 \ 

Object of Use : Evaluation ^ 1 2 3 4 5 6 8 9 



Practice Area and 

Population Administration of Justice, Decision makers 
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ROLE CONSTRUCT REPERTORY TEST 



AUTHOR: 



George A. Kelly 



AVAILABILITY: 



Kelly, G.A. The Psychology of Personal Constructs . Vols. I 
and II (New York: W.W. Norton, 1955). 



PURPOSE: 



The Role Construct Repertory Test (Rep Test) 1s a measure 
of cognitive structure which focuses on the personality. 
The purpose of the Rep Test 1s to elicit an Individual's 
personal construct system. 



VARIABLES: 



The Rep Test does not attempt to measure variables previously 
conceptualized by the researcher, it seeks, rather to 
elicit from the respondent in his own language the major 
constructs used to interpret different situations. A 
personal construct, as Kelly develops the term, 1s a channel 
of psychological movement. "In its minimum context a 
construct is a way in which at least two elements are 
similar and contrast with a third" (Kelly, 1955). Three 
elements are the minimum required for a construct; the 
two which are similar are termed the "construed"; the 
third is called the "contrast". For example, two roles may 
be seen as having in common the element "easygoing", and 
a third role may be specifically contrasted by having the 
element "hypertensive," 



DESCRIPTION : The Rep Test 1s not just a single test, but a highly flexible 
technique. A number of fbrms have developed over the years, 
each of them tailored to specific purposes. The general 
method, however, requires respondents to simultaneously 
compare and contrast elements along dimensions that the 
respondent himself generates. 



DEVELOPMENT: 



The Rep Test is based on a highly developed and formal 
theoretical framework called Personal Construct Theory 
(Kelly, 1955). 

Personal construct theory posits that man uses his cognitive 
processes to predict and control his environment. As ex- 
perience is gained, each individual learns to codify observa- 
tions into a system of interrelated interpretations, thus 
developing his own set of personal constructs by which the 
individual structures his world and learns to anticipate 
future events. The psychology of personal constructs is 
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concerned with ways to analyze and describe an individual's 

unique set of constructs as well as to discover how and 

why these sets develop and change (Bannister and Mair, 1968). 

The implications of this theory are that an individual 
responds to a situation in terms of how he uniquely 
perceives the situation. In turn, an individual's inter-, 
pretation of a situation is directly dependent upon his 
current repertoire of , person?! constructs. Therefore, the 
ability to understand and predict human behavior depends 
upon the extent to which the relevant set of personal 
constructs can be surfaced (Bannister arid Mai re, 1968). 



Kelly rejects the concept of reliability as unsuitable to 
the Rep Test; he prefers to discuss its consistency, and 
refers to studies of hospital patients and of college 
students which showed that on two administrations of two 
different but equivalent forms of the test, the average 
agreements in constructs used were 69% and 70% 1n the two 
groups respectively. The standard deviations were 6% 
and 8%. Field and Landfield (1961) found a two-week test- 
retest correlation of .79 between constructs elicited by 
the same el&itents and on of .80 between constructs elicited 
by new elements (n*80 normals). They also found that when 
subjects chose their own elements with regard for role 
titles, they repeated 72% of the role title occupants. 
Pedersen (1958) found an average agreement of 77% between 
persons chosen to fill role titles in two separate admin* 
1 strati ons one week apart (Lake, Miles and Earle, 1973). 



There are many different variations of the Rep Test. 
Basically and briefly, the technique is applied as follows: 

1. A list is drwan up of all the objects in a field 
under study. The length of the list is hot critically 
important, but the optimum number is between 10-30 
Items or subsets. 

2. The items are then transferred onto a set of numbered 
cards, each of which carries the name of a single 
object. 

3. A grid is prepared for recording the subject's re- 
sponses. The columns of the grid are headed by 
numbers which correspond to, the numbered cards 
carrying the name of the object or subset. 

4. The Interviews are carried out individually: a) 
the subject is shown the entire pack of cards and 
told to sort through thetti and remove any with 
which he is totally unfamiliar; b) three cards are 
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selected 1n a predetermined random order; and c) 
the respondent is asked "to think of a way in which 
any two of the three are similar to each other and 
different from the third" (Frost and Braine, 1967:165). 
Thus, without being led, the respondent reveals a 
construct which he uses £o categorize and differentiate 
between objects 1n the field. 

5. Having identified a dimension by which the user 
simultaneously categorizes and differentiates , 
between objects, the respondent is asked to sort 
through the cards and rate the renal oing brands 
along this dimension. This rating may be in the 
form of a paired comparison, a ranking, or a rating 
along a scale. 

6/^ Next the respondent is handed a different set of 
cards and the procedure Is repeated (1 .e. , "to think 
of a way in which any two of the three are similar 
to each other and different from the third"). In 
addition, the respondent is instructed not to repeat 
a response that was given previously, in order 
to generate a new construct. 

7. The process continues until the respondent can no 
longer think of new constructs: The average response 
is eighteen constructs per interview (Frost and 
Braine, 1967). In most cases only forty interviews 
are necessary to identify all the relevant attributes 
in a domain of interest. 



SOURCES : Bannister, D. and Mai r, J.M.M. The Evaluation of Personal 
Constructs . New York: Academic Press, 1968. 

Fjeld, S.P. and A.W. Landfield. "Personal Construct 

Consistency." Psychological Reports , Vol, 8, (1961) :127. 

Lalce, D.G., M.B. Miles, and R.B. Earle, Jr. (eds.). 
Measuring Human Behavior , New York: Teacher College 
Press, 1973. 

Pedersen, F.A. "A Consistency Study of the R.C.R.T." Un- 
published master's thesis, CoVumbus: Ohio State 
University, 1958. ( 
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CRITICAL THINKING, EDUCATIONAL PREPARATION, AND 
DEVELOPMENT OF MORAL JUDGEMENT 



Shake Keteflan 



Keteflan, Shake. "Critical Thinking, Educational Preparation, 
and Development of Moral Judgement." Nursing Research . 
Vol. 30, number 2, x (March-April 1981) : §6-103. 



Watson-Glaser Critical Thinking Test (1964) and the Defining 
Issues Test (Rest, 1974) were used with a group of nurses 1n 
nursing school to determine the relationship between 
critical thinking, level of education, and level of moral 
judgement. 



1) Critical Thinking, measured by the Watson-Glaser Critical 
Thinking Appraisal Test Form ZM (1964); 2) Moral Reasoning, 
measured by Rest's Defining Issues Test, (1976), and 3) 
Educational Preparation. This variable 1s divided into two 
categories: Skill oriented education, on a junior or 
community college level, and Professional, based on 
theory and offered by "senior college or university nursing 
program". (100) 



These variables were operational ized 1n the following manner. 
The Watson-Glaser Critical TMnklng Appraisal Test 1s com r 
posed of 100 items "that require application of cognitive' 
abilities in solving problems, interpreting and evaluating 
statements and arguments similar to those encountered 
, in early life." (100). Total'scOres, as well as five 
subscores (Inference,* recognition of assumptions, deduction, 
interpretation, and evaluation of arguments.) 

The Moral Judgement Development Measure was developed by 
J. Rest and others. Six hypothetical stories, each with a 
moral dilemma," are presented to respondents. Each story, 
in turn, contains twelve "issue statements". Each of these 
statements 1s a "distinctive way of defining and evaluating 
a social moral d1lenrna" (101). Respondents rate each issue 
for each story on a scale of importance, and then re- ranks 
the Issues ranked firsf through fourth. 

Educational Preparation was operational ized by levels of 
education gathered on personal Information sheets by 
Professor Keteflan. 
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DEVELOPMENT: 



Three hypotheses were presented by Keteflan: 1) A positive 
relationship exists between^ critical thinking and moral 
reasoning; 2) A difference exists 1n moral reasoning be- 
tween professional and technical nurses, and 3) critical 
thinking and aducatlonal preparation together predict 
greater variance 1n moral reasoning than either considered 
separately. j 



RELIABILITY/ 
VALIDITY: 



The Moral Judgement Development Measure was tested for 
validity with multi-aged groups of students, from ages 
fourteen through Ph.D. students. One way ANOUA of scores 
across the four groups showed an F value at or beyond the 
.01 level of significance. Reliability, as determined by 
test-retest correlation data, was .81. 

The Critical Thinking Measure reliability, consisting of 
odd-even split half reliability coefficients, range from 
.77 to .83. Construct validity, 1n the form of correlations 
with various Intelligence and comprehension tests, ranges 
from .65 to .73. 



ADMINISTRATION: 



One hundred and fifty-eight registered nurses were given 
packets containing the Critical Thinking Measure Test, 
the Defining Issues Test, and a personal Information' sheet. 
Seventy-nine responses were analyzed. All three hypotheses 
were upheld. 



SOURCES : Watson, G. and E.M. Gl*ser. Critical Thinking Appraisal 
Manual . New York: Harcourt, Brace and World, 1964. 

Rest, J. Manual for the Defining Issues Test . Minneapolis, 
The Un1vers1ty~o? MTnnesots, 1974. (mlmeo). 
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STUDY PROFILE 

• DIMENSION DESCRIPTION CODE 

0n1t of Analysis 4J _ (pi)o2 03 04 05 06 07 08 09 

Individual v — ' 

Sampling ••••• Purposive Ol(o2)03 04 07 08 09 

Design ... Quasi-experimental without 01 02 03 04 ©06 07 08 09 

control groups and/or randomi- ^ "° u ' 
r ati on 

Research Methods........ Questionnaires 01 02 03 04 05® 07 08 09 

Analytic Methods Multivariate Analysis 1 2" 3 ® 5 8 9 

Analytic Focus Non-qualitative ..." 10 8 9 

liability Reported 1„©8 

Validity Reported 1 (?) 8 " 

Definition of Use, Conceptual 10389 

Object of Use Study 1 (?) 3 4 5 6 8 9 

Practice Area and „ u 

Population... Health, Nurses 

» 

o ion 
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INSTRUCTIONAL CHAN6E INVENTORY 



AUTHOR: 



Harry J: Knopke 



AVAILABILITY: 



Knopke, Harry J. "Instructional Change Inventory 1 
Office of Educational Development. College of 
CommunUy Health Sciences, University of Alabama, 
P.O. Box €291, University, Alabama 35486. 



PURPOSE : The instrument was designed to identify the extent to 

which recommended changes involving the teaching-learning 

process had been implemented with major program changes. 



VARIABLES: 



Comnitment to change, extent' of perceived implementation 
of change. 



DESCRIPTION: 



Twenty ftepis in a seven-point Likert-type format measure 
both commitment to change (to h great extent— to a slight 
extent) and perceived implementation of change (definitely 
is being implemented— definitely is not begtn implemented). 

The respondents are asked to indicate the extent to which 
they feel committed to implement each of item in their teach 
ing activities, and write the appropriate number, using the 
following scale:. 



Committed to 
a great extent 



Committed to a 
slight extent 



Delineate students' cognitive, attitudinal, and psycty 
motor objectives for each course in terms of overall i 
goals and objectives in the curriculum. 

V 

Describe student performance in a course in terms 
of group achievement to allow for academic compe- 
tition and rank ordering. 



DEVELOPMENT : Both a review of the literature and an examination of 

prograauiQCuments of schools involved in the study resulted 

in a list of forty statements describing either innovative 
or traditional instructional practices. The professional 
health care educators reviewed the items for relevance to 
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RELIABILITY/ 
VALIDITY: 



ADMINISTRATION : , 



health care education and to correct omission. The result- 
ing thirty-two Items were reviewed for content, the remaining 
twenty-eight Items were administered to full-time under- 
graduate faculty members 1n two nursing schools. Ninety- 
seven completed Instruments were returned. Items with 
Item subscale score correlation of less than .34 were 
discarded, leaving twenty items, of which eleven represent- 
ed Innovative Instructional practices and nine- traditional 
practices. 



For the revised Instrument, Internal consistency was 
assessed from the data obtained from the Instrument 
development studv. Coefficient alpha was .77 for first 
part (commitment) and .73 for the second part (Implement- 
ation). The Instrument was administered to faculties of 
the school of nur$1ng (n-38; 84% return rate), school of 
medicine (n»40; 72% return rate), and school Qf pharmacy 
(n»27; 84X return rate). Coefficient alpha was calculated 
on the total sample of 104; 1t was .77 for first part and 
.82 for the second part. 

The factor analysis of the conmltment data have construct v 
validity Implications. 



The test 1s self-adm1n1stered. Time required for completion 
1s not specified, although 20 minutes appears to be 
sufficient for most respondents. 

T\\e Inventory yields two subscale scores, degree of 
commitment and perceived degree of Implementation. 



Knopke, Harry J. "Instructional Change 1n#tealth Care 
Education." Nursing Research , Vol. 25, (1976) :262-67. 
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STUDY PROFILE 

. ..... * 

DIMENSION DESCRIPTION CODE 

™t of A Wl ys1s " j^^gg^j 01 02 03 04' 05 O^gpS 09 ' 

Sampling 01 02 03 04 07 08 09 : 

Purposive 

Design Uncddable 01 02 03 04 05 06 07 08 09 

Research Methods Questionnaire 01 02 03 04 05 06 07 08 09 

Analytic Methods Ratings 1 2 3 4 5 8 9 

Analytic Focus '••••Non-qualitative . .12 8 9 

Reliability 12 8 

Reported 

Validly Not reported 128 

Definition of Use Instrumental 1 2 3 8 9 

Object of Use Evaluation ' 1 2 3 4 5 6 8 9 

<<\ 

Practice Area and 

Population Nursing Education Research, Faculty 1n Nursing 



ERIC 



91 



INFORMATION UTILIZATION INSTRUMENT 



AUTHOR: Judith Larsen 



AVAILABILITY: 



American Institutes for Research 1n the Behavioral Sciences, 
P.O. Box 1113, 1791 Arastradero Road, Palo Alto, CA, 94302. 



PURPOSE : , The purpose of the IUT 1s to determine the nature and extent 

of utilization of Information passed from experts to 

potential user. 



VARIABLES: 



Util1zation/Non-ut1l1zat1on of information. 



DESCRIPTION: 



This procedure for determining utilization assumes that 
Information 1s presented to potential users by a consultant 
or expert. The expert may present Information 1n any of 
severl forms: descriptions of research findings, explicit 
action recommendations, general suggestions, a series of 
Ideas, etc. 



The fexpert meets with potential users and presents the 
Information to them. This may take the form of a con- 
sultation, a site visit, a phone conversation, etc. The 
critical event is that specific Information passed from 
expert to potential user must be Identified. The technique 
used by the author was to have a trained observer note 
specific Information provided by a consultant. 



DEVELOPMENT: 



Not reported. 



RELIABILITY/ 
VALIDITY: 



ADMINISTRATION: 



Not reported. 

The application the author used w1^h this Instrument was 
consultation. Each consultant was accompanied by a trained 
observer who noted specific suggestions provided by the 
consultant. A 11st of suggestions was left with the users 
and a copy retained by the observer. At two points in time, 
four months and eight months, following the orglnaT visit, 
the list of suggestions were used to form the basis of 
follow-up telephone Interviews with users. The purpose of 
the Interviews was to determine the nature and extent of 
utilization. 
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Interviews were conducted with several potential users in 
each organization. The Interviewer presented each suggestion 
Individually. Users were asked what, 1f anything, had been 
done with each suggestion. If the suggestion had been 
discussed, the Interviewer attempted" to find out what 
decision had been made. If the users decided not to do 
anything, the Interviewer attempted to learn their reasons. 
If users decided to Implement the suggestion, the Interviewer 
attempted to learn what specific steps they had taken and 
how the Implementation process was proceeding. Thel Inter- 
viewer continued probing until the appropriate category for 
the response was clear. Responses were then asslgrted to 
one of seven categories of use or non-use. Some examples 
of the categories are: 

(1) Information considered and rejected. Some discussion 
took place, but the information was no longer\beiifa 
considered. \ 

" - 1 
(4) Steps taken toward" implementation. Although the infbr- 
rnatioh had not been implemented, the decision to do so 
had been made and steps toward it had been taken. 

(7) Implemented and adapted to fit the user's needs. 
Certain features of the information were modified 
or adapted to fit the local situation. 
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' STUDY PROFILE . ' 

•V ' ' *• 

DIMENSION DESCRIPTION ' CODE' 

■ ■ ■ » ■■ • * 

Unit of Analysis........ Indiv1 ^ ual *™ (oi))2 03 04 05 06 07 08 09 

Sam P lin 9 Unavailable- " 01 02 03 04 07 @09 

Desi 9 n •••• Unavailable 01 02 « 04 05 06 07 <® 09 

Research Methods... * lt1plt Weth ods 01 02 03 04 05 09 

. Analytic Methods........ Not availabl ^ 1.2 3 4 509 

. ' Analytic Focus Non-qualitative 1 1 © 8 9 

Reliability. - Not reported (O 2 8 



Validity. • Not reported (T) 2 8 

' 1 — • ' 

Definition of Use Instrumental f (^)* 3 8 9 

Object of Use Person-embodied Innovation 1 1 2 3 4 5j6 8 9 

■ — — f » . . 

Practice Area and ^ 
Population.... Adaptable to any practice area, Untargeted 
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PROFILE OF ORGANIZATIONAL CHARACTERISTICS 



^PKUFILI 



AUTHOR: 



Rensis Likert 



AVAILABILITY: 



Likert, Rensis. The Human Organization . New York: 
McGraw-Hill, 1967. 



PURPOSE: 



The instrument was designed to measure the prevalence 

of defined "management systems" within organizations 

1) "Exploitative Authentative", 2) "Benevolent Authentative," 

3) "Consultative," and 4) "Participative." 



/ - 



-VARIABLES: 



Variables measured fall into eight categories: 1) Leader- 
ship process used, 2) Character of Motivational Forces, 3) 
Character of Communication Process, 4) Character of 
Decisionmaking Process, 6) Character of GoaV Setting or 
Ordering, 7) Character of Control Process, and 8) 
Performance Goals and Training. 



DESCRIPTION: 



DEVELOPMENT: 



Fifty one items falling under the eight categories above 
are answered by respondents with modified Likert-type 
scale values. (20 point scale). 



Not reported.. 



RELIABILITY/ 
VALIDITY: 



Split-half reliabilities of .81, .97 and .99 were reported 
for various samples. No other information was' reported. 

No validity coefficients were reported. > 



ADMINISTRATION: 



SOURCES: 



The self -administered questionnaire takes between thirty 
and forty minutes to complete. 



U' 



Not reported. 
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MEASURING LEVELS OF USE* OF THE INNOVATION 



AUTHOR: 



AVAILABILITY: 



Susan F. Loucks, Beulah W. Newlove and Gene E. Hall 



Loucks, Susan F. t Beulah W. Newlove and Gene E. Hall 
Measuring Levels of Use of the Innovation :, A Manual 
Trainers. Interviewers and Raters. 



for 



- — . ,. Austin: Research and 

Development Center for Teacher Education, University of 
Texas at Austin. 



PURPOSE : The Level? of Use Scale seeks to describe behaviors of 

Innovation users. It defines operationally the various 

"states'' or categories of Innovation user behavior. 

a 

VARIABLES : Eight discrete levels of use of an Innovation are described: 

* 0~jNon-Use: user has little or no knowledge of the Innovation); 

l~(0rientat1on: user 1s gathering Information about the 
innovation); 2— (Preparation: the user 1s preparing for 
first use of the Innovation); 3— (Mechanical Use: user 
directs attention to day-to-day use of the Innovation); 
4a~(Rout1ne: use of the Innovation 1s stabilized); 4b-- 
(Refinement: user varies the use of the Innovation); 5— 
(Integratlon: user combines use of Innovation with 
colleagues to achieve a collective Impact); and 6— (Renewal: 
user reevaluates the quality of use of the Innovation). 

DESCRIPTION : The Levels of Use Scale matches the eight discrete levels 

of use outlined 1n "variables" above with seven categories 

of Innovation adoption and Implementation: "Knowledge 1 ! 
(what the user knows about characteristics of tjie Innovation 
and how to use 1t; "Acquiring Information" (soliciting the 
Information about the Innovation); "Sharing" (discussing the 
Innovation wfth others); "Assessing" (examining the potential 
or actual use of the Innovation); "Planning" (designing and 
planning steps for adopting and implementing the Innovation); 
"Status Reporting" (outlining personal feelings about the 
use of the innovation); and "Performing" (carrying out the 
actual operationalization of the Innovation). 



DEVELOPMENT: 



The Levels of Use Scale 1s part of the larger research on 
Innovation adoption and Implementation being carried out 
*at the Research and Development Center for Teacher Education 
at the Un1verst1y of Austin. The theoretical model of which 
the scale 1s a part, 1s one which views change "as a process, 
not as an event", and examines Innovations as a change pro- 
cess, focusing on the Individual's feelings about the innovation 
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(the Stages of Concern Scale— see page 60 ), the rate 
and way 1n which the innovation 1s used (Level of Use Scale), 
• and the variations of the innovation which are arrived at 
by various individuals as the Innovation is disseminated 
(the Innovation Configuration Scale— see page 67). 

RELIABILITY/ 

VALIDITY : Interrater reliability is estimated by assigning each 

interview to two raters. If the two don't agree, a third 

rater is used. Loucks, Newlove and Hall report that 1n 
the Initial use of LoU, 36% of the interviews required 
a third rater. 

Rater reliability was determined by randomly pairing raters. 
The percent of agreement each rater obtains with the 
other is an Index. X* tests were used to determine whether 
significant differences appeared between raters. The 
weighted mean of all the raters' percent of agreement equals 
the percent of interviews agreed upon by two raters'. This 
measure serves as an index of reliability. In the Initial 
use of the LoU, 65% to 70% of tapes were agreed upon by 
two raters. 



ADMINISTRATION : Data for the Level of Use Scale is gathered through an 

1 Interview, which is generally taped for actual ratings 

at a later date. Interviewers and Raters— those who 
actually decide at what point of the LoU the respondent 
1s ,based on taped answers— are often not the same Individuals. 
Interviews vary from ten to thirty-five minutes. The 
principle objective of the Interview 1s to gather sufficient 
Information to assign a Level of Use. Based on the results 
of the rat1ngs--wh1ch are descriptive values pinpointing 
the level of "use— the researcher may assign each Individual 
a Level of Use. Rater judgements are subjective, based on 
responses. 



SOURCES : Hall, Gene, Susan Loucks, William Rutherford and Beulah 

Newlove. "Levels of Use of the Innovation:. A 

Framework for Analyzing Innovation Adoption." Journal 
of Teacher Education . Vol. 26, number 1 (Spring 1975): 
5-7. 

Hall, Gene, R.C. Wallace and W.E. Dosset. A Developmental 
Conceptualization of the Adoption Process Within 
Educational Institutions . Austin: Research and Develop- 
ment center tor Teacher Education, University of Texas 
at Austin, 1973. 

Ebel, R.L. "Estimation of the Reliability of Ratings." 
Psychometrika . Vol. 16 (1951): 407-24. 
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STUDY PROFILE 



DIMENSION ( DESCRIPTION . CODE 

Unit of Analysis Individual (Ol)o2 03 04 05 06 07 08 09 

Sampling Purposive 01 (02)03 04 07 08 09 * 

Quasi -experimental without 

Design control groups and/or 01 02 03 04(05)06 07 08 09 

randomization „. ^_ ^-^ 

Research Methods... ^ Interviews ' S 01 02\03 04(65)06 07 08 09 

Analytic Methods Bivari ate Analysis 1 2(?) 4 5 8 9 

Analytic Focus.,... Non-qualitative 1 l(z)8 9 

Reliability Reported 1 © 8 

Validity....: Reported 1 (2) 8 

Definition of Use Instrumental Q 2 3 8 9 

Object of Use Pol Icy/Management Innovation 1 2 (fD 4 5 6 89 

Practice Area and 

Population Education-practitioners, policymakers 
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STRATEGIES FOR KNOWLEDGE USE AND SCHOOL IMPROVEMENT 



AUTHOR: 



Karen Lojrtr; Sheila Rosenblum and James Moll tor 



AVAILABILITY: 



Louis, K.S., Rosenblum, S. and J. A. Mollton. Strategies 
for Knowledge Use and School Improvement . Cambridge, MA: 
Abt Assoc1ates7^uTy~l§8X 



PURPOSE : The purpose of this research project 1s to study the R&D 

Ut1l 1 zat1 on v Program established by The National Institute of 

Education wh,1ch was designed to test whether school -level 
practices could be Improved by making external resources 
available to school- personnel. 

VARIABLES : The authors developed a general model which guided the 

: study and analysis of the Impacts of- the RDU program. The 

model 1s divided Into three sections: 1) local conditions 
which Include concepts and variables derived largely from 
the "Innovative organizations" tradition; 2) Intervention 
strategies which map the RDU approach onto the "strategies 
of change" tradition, and which categorize the strategies 
employed by RDU Into three different types— Information, 
technical assistance and an Internal problem solving process 
and; 3) proximate and distal outcomes , both those Intended 
by NIE and/or the projects and the unintended outcomes. 

DESCRIPTION : This study addressed six major 1ssues\ 

* how relationships are managed anxjng\ft^aHous agencies 
that possess the expertise and resources tol^lp local 
schools solve problems; 

* to what degree an Intervention program such a/ RDU can 
help schools overcome barriers to successful problem 
solving (such as limited access to Information or lack 
of planning skills, etc.); 

* to what degree the products of educational R&D are 
relevant to the problems and contexts of local schools; 

* what the Impact 1s of the products of/educational R&D 
once they have been adopted and Implemented; 

* what factors contribute to the Institutionalization of 
the RDU approach within a variety of organizations; 
and 
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how linking argents coordinate the flow of external 
resources to schools, and whether this helps the 
schools solve problems. (15) 



DEVELOPMENT: 



The study was designed so that each data source could 
provide different types of information about the charac- 
teristics of the school, and about institutionalization 
of the process and product. The teacher questionnaire 
emphasized teacher assessments of materials, the process 
and the impacts of the activities and process on the school. 
The site visits, on the other hand, emphasized capturing the 
nature of the intervention at the school level, particularly 
the major features of the problem-solying process at 
various stages, and the level of activity of various key 
actors, both inside the school (principals, teachers, etc.) 
and outside (field agents and others). There was some 
overlap in items and topics between instruments, but the 
strategy was to develop an information "division of labor" 
based on the knowledgeability of the>espondent/observer 
about certain topics, and the need to develop both site- 
visit protocols and questionnaires that were not overly 
burdensome. (42) 



RELIABILITY/ 
VALIDITY : 



Not reported. 



ADMINISTRATION: 



Data- were collected in faGe-to-face focused bat unstructured 
interviews at fifty-one sites. Case studies were written 
on forty-six sites, five of which also received site visits. 
In addition, mail surveys were sent to principals and 
teachers at participating schools. 
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STUDY PROFILE 

f 

DIMENSION 1 DESCRIPTION ^ CODE 

Unit of Analysis Organization 01 02 03V^)05 06 07 08 09 

Sampling Purposive 0l(§^ 03 04 07 08 09 

Case studies without ^ 

Design. explicit theory \0j)02 03 04 05 06 07 08 09 

Research Methods.. Multiple methods 01 02 03 04 05 06^07)08 09 

Analytic Methods , Multivariate analysis 1 2 3 Q) 5 8 9 

Analytic Focus Qualitative iC?) 2 8 9 

Reliability (D 2 8 

Not reported ^ 

Validity CO 2 8 

Not reported V 

Definition of Use _ 4 ^ _ , (l) 2 3 8 9 

Instrumental ^ 

Object of Use — Uncodable ■ 1 2 3 4 5 6 8© 
Practice Area and 

Population r . , . . . \. 

r Schools, Linking agents 

» 
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PERSONNEL RELATIONS SURVEY 



Joseph Luft and Harry Ingham 



Luft, Joseph and Harry Ingham. "Personnel Relations Survey". 
Teleometrics, Inc., 2200 So. Post Oak Road, Suite 424, Houston, 
Texas 77027. 

The PRS is designed to determine how managers monitor and 
control the flow of information between themselves and 
others. It is des-igned to assess the understanding and 
behavior of managers in their interpersonal relationships. 



Personal relations with others Is divided into "exposure" 
and "feed back" factors. These could be measured by 
variables as "Relationship with employees," "Relationship 
with colleagues," "Relationship with supervisors." 



The survey is a sixty item inventory, with two alternatives 
reactions of handling the situation described in the item. 
Respondents are asked to choose what they would normally 
do in that situation, rather than what should they do. Fof\ 
each item, there are five points that could be distributed! 
in six possible combinations for responding to the pair \ 
of alternatives presented with each inventory item. Here \ 
5 - "completely characteristic of the respondent", 4 - 
"somewhat characteristic," and 0 ■ "completely uncharacteristic 
while 3 - "alternative A being more characteristic of the/ 
respondent than the alternative B, and 4 is reverse of / 
condition for 3. 

One example of items are: 

1. If an employee of mine had a "personality conflict" with 
a manager of another department with whom it was important 
that he cooperate in order for the work to get done, 
I would: 

A. Tell my employee that I felt he was partially re- 
sponsible for any conflicts with this other person 
and try to indicate how the other person was being 
affected by him, 

or 

B. Not get involved for fear that I wouldn't be able 
to continue to get along with them once I had 
entered it in any way. 
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DEVELOPMENT: 



The Personnel Relations Survey is based on a model of 
inter-personal relations called the Johari Window; this 
model, 1n turn, was developed some years ago by Dr. Joseph 
Luft and Harry Ingham for their programs in group dynamics 
.training. By modifying slightly their earlier work it is 
possible to treat the Johari Window as an information flow 
model and to measure the tendencies of individuals to 
facilitate or hinder the flow of interpersonal information. 



RELIABILITY/ 
VALIDITY: 



The instrument was developed in strict accordance with 
accepted principles of psychological test construction. 
In the item-analysis using a sample of 100 or more subjects, 
each item is correlated against its criterion score, and no 
items are used that do not significantly correlate. After 
item-analysis, a test-retest reliability 1s made, and if 
the instrument is stable enough, as indicated by criterion 
of .60 or better, it is then subjected to a construct 
validity check. Construct validity by and large is accept- 
able for training instruments, but not acceptable for 
research or selection devices. 



ADMINISTRATION: 



It is a self-administered questionnaire, without any time 
limit. The scoring is done by the individual taking the 
inventory. 



SOURCES; 



N/A 
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STUDY PROFILE 



DIMENSION DESCRIPTION ; CODE 

Unit of Analysis individual ®° 2 03 04 05 06 07 08 09 

Sampling Purposive ' 01@03 04 07 08 09 

Descriptive case study with ^—v ! 

Design explicit theory 01(1)2] 03 04 05 06 07 08 09 



Research Methods Questionnaires 01 02 03 04 05(6(f))7 08 09 

— \ 

Analytic Methods Codes *©3 4 5 8 9 



Analytic Focus Qualitative • (1)2 8 9 



Reliability 

Not reported 

Validity 

Not reported 

Definition of Use _ 

Instrumental 

Object of Use 

Management innovation 

Practice Area and n . . u 

PoDulAtlon Business Management, Managers 
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POLICY IMPLICATIONS ANALYSIS 



Doren Madey and A. Jackson Stenner 



Madey, Doren and'A. Jackson Stenner, "Policy Implicates 
Analysis: A Methodological Advancement for Policy Research 
and Evaluation. " Paper presented at 1980 meeting of 
Education Research Society, Arlington, Virginia. 



PIA 1s a six-step tethnlque designed to clearly articulate 
information requirements of information users" at both 
national and local levels as evaluations of policies or 
programs are undertaken. 



Variables are the types of information utilized by defined 
personnel 1n any articulated problem area. This Information 
may be as broadly or narrowly defined as possible, depending 
1n large measure on 1) the nature of the Issue or problem ■ 
Investigated, and 2) the degree of specificity desired by 
the researchers and/or actors. 



The PIA is similar in background, method, and purpose 
to the delphl technique, scenario writing, (a la Herman 
Kahn), and various conferencing techniques which have as 
their goal the structured and systematic explication of 
problem formalities by various actors and stakeholders 
1n a given problem domain. The technique Itself has six 
steps. f 

(1) Once the problem area has been Identified, re- 
searchers create "statements of hypothetical, bat 
theoretically possible findings which could result 
frqm the evaluation...", ranging from "relatively 
straightforward" to "relatively unexpected". 

(2) On the basis of these scenarios, packets are made 
up containing introductory descriptions of the 
PIA process, the scenarios, and a series of 1n- 
structloas for respondents allowing them to respond 
to the scenarios as well as the program being 
evaluated. 

(3) Respondents-policymakers and other "information 
users" ar6T*ent1f1ed by the Research team. 

(4) Respondents are asked to complete the packet de- 
scribed in (2) above. Each scenario 1s ranked 
according to the respondents approximation of the 
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Hkellness of eachWcenario occurring, preferred 
policy actions with respect to the program being 
evaluated, and information requirements for further 
analysis. 



(5) Responses are analyzed tMjgh quafi 
reduction techniques, and \ 



itative data 



(6) Responses are used to create 
program evaluation. 



hypotheses concerning 



DEVELOPMENT: 



RELIABILITY/ 
VALIDITY: 



PIA is based on the Delphi TechniqueXand Scenario Con- 
struction. While the process is generally not iterative, 
it is used to uncover "diverse expectations .and information 
needs emanating from different ^perspectives. 11 

\ 



No reliability or validity reported. 



ADMINISTRATION: 



PIA is a self-administered questionnaire and associated 
forms. No times are reported. 



SOURCES : Stenner, A.J., and D.L. Masley. Policy Implications Analysis , 
Exercise 1, "Design for the Evaluation of the State 

Capacity Building Progra^ in Dis semi nation." Durham, 

N.C.: NTS Research Corporation, 1976. 

Madey, D.L. and A.J. Stenner. ^Policy Implications Analysis: 
.A Methodological Advancement for Policy Research and 
Evaluation." Paper presented at the 198a meeting of the 
Evaluation Research Society, Arlington, Virginia. 
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STUDY PROFILE 

DIMENSION DESCRIPTION CODE 

~"~ : r— 

Unit of Analysis 0tner ( 1o !r a1 * nd national Q1 02 03 04 05 06(07^08 09 

policymakers) » SZJ 

Sam P" n 9 Purposive - 01 (oF)03irOT 08 09 . 

Design ex^ci^heSry* 0l ® 02 04 05 06 07 08 09 

Research Methods Questionnaire 01 02 03 04 OS*® 07 08 09 

• , . . M A . , Unsystematic, Empirical /TA « 

Analytic Methods generalisations (^ 2 3 4 5 8 9 

Analytic Focus...:.' Qualitative (£) 2 8 9 

Reliability... No estimate -0 2 8 

Validity No estimate (pa s 

Definition of Use Symbolic ~ 1 2(3^8 9 

Object of Use Project Report/Evaluation (T)2 3 4 5 6 8 9 

Practl qfe" Area and 9 

Population General 
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REMOTE ASSOCIATES TEST 



AUTHOR: 



Sarnoff Mednick and Martha Mednick 



AVAILABILITY: 



Mednick, Sarnoff and Martha Mednick. Research 1n Personalit y. 
New York: Holt, Rinehart and Winston, T9bJ; : 



PURPOSE : The Remote Associates Test is designed to measure the 
ability of individuals to associate ideas that are seemingly 

"remote" from one another. This ability is taken to be an 

indication of creativity. 



VARIABLES: 



Variables are" associations which respondents make between 
ideas or concepts considered to be "mutually remote". 



DESCRIPTION: 



The RAT is composed of thirty items, each consisting of 
three words drawn from groups determined to be mutually remote- 
that Is, non-associative, by the authors. Respondents are 
asked to "think of a fourth word which can be associated 
with each of the three in the cluster, and which thus 
serves to associate each of the words. One cluster is: 

Cookies sixteen heart 

with a probable fourth being "sweet". 



DEVELOPMENT: 



Words used in the RAT were developed by utilizing words 
found to have a low probability response 1n standard word 
association tests such as the Kent-Rosanoff Word Association 
Test or the Thorndl ke-Lorge Word Count. By thus attempting 
to ..assure low levels of association, clusters of words could 
beconstucted which had zero levels of association. 



RELIABILITY/ 
VALIDITY: 



Corrected odd-even, reliability for the test was .92 for 
women, and .91 for men 1i\ tests at various colleges* 

Criteria validity was determined by measuring results of 
the RAT against Instructor's assessments of twenty students' 
creativity at a college of -architecture. The correlation- 
between students' RAT scores and the Instructors' rating 
was .70 (p <.01). 
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ADMINISTRATION: 



There 1s a forty minute time limit generally imposed in the 
thirty item self administered questionnaire. Scores are 
computed by adding the number of correct numbers. 



SOURCES : Mednick, M. "Research Creativity in Psychology Graduate 

Students." Journal of Consulting Psychology. Vol. 27. 

(1963): 265-65: 

Wallach, M.A., and N. Kogan. "A New Look at the Creativity- 
Intelligence Distinction." Journal of Person ality. Vol. 33. 
(1965): 348-69. ' 



Wallach, M.A. and N. Hogan. Modes of Thinking In Young 
Chil dren : A Study of the Creativity- Intelligence 
i net ion . /New York: Holt, Rinehart and Winston, 



Distincti 



1965. 



9 

ERIC 



109 



STUDY PROFILE 

" ■ v i ' 

«* 

DIMENSION DESCRIPTION , CODE 

Un1|? of Analysis Individual > @02 03 04 05 06 07 08 09 

Sampling Purposive. 01<@03 04 07 08 09 

Quasi-experimental without /~\ 

Design control groups and/or random- 01 02 03 0* 05)06 07 08 09 

ization w 

Research-Methods Questionnaire 01 02 03 04 05 @° 7 08 09 

Analytic Methods Bivariate Analysis 1 2 04 5 8.9 

• Analytic Focus "•••Non-qualitative 1 1 (J) 8 9 

. Reliability Reported 02 8 

Validity Reported 

Definition of Use Conceptual 103 8 9 

Object of Use Study 1 (J) 3 4 5 6 8 9 

Practice Area and „ ' . . 

Popul atlon. General , Students 
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MEETINGS 



1 



Matthew B. Miles 



Miles, Matthew B. "Meetings" 1n Dale Lake, Final Report 
Cooperative Project for Educational Development. 
Washington: U.S. OfTTce of Education, 1970. 



The Instrument 1s designed to measure group problem-solving 
adequacy. 



The thirty-seven Items were written to cover all stages 
of, problem- solving, plus several continuing functions 
(summarizing, process analyzing, participation), plus 
Items dealing with positive and negative climate. The 
stages of problem-solving supposedly sampled by items are: 
agenda clarity and control; problem definition; diagnosis; 
solution generation; solution discussion; decision-making; 
implementation; follow-up. Solution-adequacy is also 
measured. 



Each item describes a function usually Involved in problem- 
solving in a particular work group of .which the respondent 
1s a 4 member. The respondent is asked to Indicate the 
frequency of Its occurrence on a seven^pofnt scale. A 
typical Item is: 

"People ask why the problem exists, what the causes 
are." 

The respondent indicates how frequently this hanpens, from 
very frequently to very infrequently. An earlter version 
uses "very typical" to "not typical at all." 



The instrument 1s one of twenty created for the Co- 
operative Project for Educational Development (COPED). 
As/^uch, it was Initially field-tested on a small number 
of teachers and principals (n*150) and then Inserted in 
a package of twenty Instruments and administered to more 
than 3,000 adults 1n twenty-one school systems ranging 
from rural to metropolitan. The entire package was ad- 
ministered at two points in time during one school year. 
Each Instrument was then lifted and further developed 
by Interested authors. 
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Test-retest studies to date have yielded average Item 
reliabilities of .60. The average 1tem-1ntercorrelat1on 
was about .30. The positive sum correlates .89 with total 
score, and negative sum .90 with total score. 

Construct validity of the Instrument was explored through 
the use of four separate factor analysis yielding three 
factors which have been named -Decision-Making Effectiveness 
Problem-Solving Adequacy, and Commitment (Involved 1n 
meeting). The criterion for Including an Item 1n a factor 
was that 1t must have a .56 or better loading 1n at least 
three out of four analyses. The four studies Included 
sample sizes as follows: 625, 48, 491, and 122 (all 
adults employed 1n school systems in the COPED study). 



Scoring Instructions Indicate that 15 of the Items are 
worded 1n a positive direction; the sum of responses on 
these can be seen as a score Implying the presence of • 
behavior Indicating problem-solving adequacy such as, 
"when a decision 1s made, 1t 1s clear who would carry 
1t out an^d when." 

The remaining Items are worded 1n a negative direction, 
but reverse-scored. This sum Implies relative absence 
of behaviors usually thought to represent adequate 
problem-solving. The sum of these two scores 1s used 
as an over-all score representing problem-solving adequacy. 

Miles, Mathew B. "Meetings" in Dale Lake, Final Report 
Cooperative Project for Educational Development . 
Washington: U.S. Office of Education, 1970. 
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AUTHOR : 



AVAILABILITY: 



PURPOSE : 



VARIABLES: 



\ 



DESCRIPTION: 



DEVELOPMENT: 



RELIABILITY/ 
VALIDITY: 



ADMINISTRATION: 



SCIENCE IMPACT ON LEGISLATIVE DECISION MAKING PROCESS 



Douglas E. Mitchell, 



Mitchell, Douglas E. "Social Science Impact on Legislative 
Decision Making: Process and Substance," Educational 
Researcher , (November 1980):9-19. 



This study was conducted in order to develop a clearer 
understanding of the impact of social science on policy 
making. This study specifically focused on the role of 
state legislatures in formulating basic educational policies. 



First, the study attempted to identify, define and measure 
distinct stages in the development of major issues. Three 
distinct stages were hypothesized: 1) articulation stage- 
translating diffuse demands into specific proposals for 
legislative enactment; 2) aggregation stage-which consists 
of compromising, scruti^ajzTn>-Jf^*^4 implications of a proposal 
etc.; 3) allocation, concerns the priority of the proposal. 

Second, the study attempted toHcrentify the types of resources 
that are important in controlling the decision making process. 



Each respondent. was asked to: 1) describe recent legis- 
lation in one or more of six predetermined issue areas; 
2) identify the key actors who formulate and influence 
these legislative decisions; 3) discuss how the legislature 
is influenced by these actors; 4) define "the issues" being 
resolved as legislation is enacted; and 5) identify the 
resources utilized to control or influence the decision- 
making process. (10) 



Not reported. 



Not reported. 



Data for the study were collected from 160 key legislative 
policy influentials in the states of Arizona, California 
and Oregon. Tape recorded interviews with the respondents 
(lasting twenty minutes to li hours) covered the develop- 
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ment af six educational policy Issues: basic skills, 
personnel (Including collective bargaining) .finance, 
special education,, governance reform, and ca£e|orical aid 
programs. 

Respondents Included members of education and fiscal 
committees, elected legislative leaders; personal committee 
or chamber staff consultants; lobbyists representing 
educational Interest groups and professional staff members 
from state departments of education and the governor's 
executive staff. 

A content analysis- of approximately one-third of the verbatim 
transcripts was used to develop a general framework for 
Interpretation of the data. The general framework was then 
checked against the remaining Interview tapes for consistency 
and evaluation. (9-10) 



SOURCES : Mitchell, Douglas E. Shaping Legislative Decisions : 

Education Policy and the Social sciences (Lexington, 

Mass: D.C. Heath an? £o7, Lexington Books, 1981). 
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STUDY PROFILE 

• - • ~y 

DIMENSION DESCRIPTION CODE 

Unit of Analysis Other (legislature) 7)1 02 03 04 05 06 ©08 09 

Sampling Purposive Ol(()2)o3 04 07 08 09 

"«<«« ••• .SSSTjlgg gfe ®« «3 04 05 06 07 08 09 

Research Methods ........ Inte rviews ' 01 02 03 04 © 06 07 08 09 

Analytic Methods Empirical generalization 

02 3 4 5 8 9 

Analytic Focus Non-qual1tat1ve ' 1 G) 8 9 - 

Reliability Not reported Q 2 8 

Validity Not reported © 2 8 

Definition of Use....... Instrumental G) 2 3 8 9 

Object of Use Uncodable 12345 6 8 

p£™ilt?nJ rea ^ Ke y legislative policymakers 1n"the field of education 
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INFORMATION PREFERENCES OF STATE EDUCATION POLICYMAKERS 



AUTHOR: 



Linda J. Nelson ancj Michael „W. Kirst 



AVAILABILITY: 



Nelson, L.J. and M.W. Kirst. Institute for Research on 
Education Finance alnd Government. CERAS Building, School 
of Education, Stanford University, Stanford, CA 94305. . 



PURPOSE : The purpose of this study was to determine: (1) which 
modes and sources of information state education policy- 
makers like best and why, (2) whether policymakers use 
research 1n their work-related activities, and (3) how 
to improve research dissemination strategies in order to 
better reach target audiences. 



VARIABLES: 



DESCRIPTION:, 



Several sets of variables were measured In this study in- 
cluding: (1) the organizational sources of information 
which are considered to be important in the worlt of re- 
spondents; (2) the most valued information source of the 
respondent; (3) the attributes that make information useful 
to respondents; (4) the modes of oral and written and vis- 
ual communication used by respondents; (5) stage in the 
policy cycle; (6) barriers that block research use; (7) 
perceived future information needs. 

f ' - 

This six page mail questionnaire consists of various types 
of scales^ regarding information usager Respondents mark 
each item on a three-point scale indicating how often an 
information source is. used and how valuable the source is 
in forming opinions about issues. Information sources are 
listed and respondents are instructed to rate with the 
following scale: 

I use this source of information 

!) often 
) occasionally 
) . rarely 

The information I obtain from this source is 

( ) crucial 

of some use 
of little use 



DEVELOPMENT : The Policy Communication Research Project at the Institute 
for Research on Educational Finance and Governance has 
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developed a research program with the aim of answering 
questions about research/ policy linkages. School finance 
and special education were selected for the initial phase 
of this research because these issues seem to repres*jvt- 
two different configurations of policy communities. 



RELIABILITY/ 
VALIDITY: 



None reported. 



ADMINISTRATION: 



The six page questionnaire was mailed' to a purposive sample 
of 493 policymakers. Individuals key to the policymaking 
process within each state and for eacn issue covered were 
carefully selected. This was to ensure that the responses 
reflected the preferences of state policymakers most active 
in school finance and special education arenas. 



*3 



9 

EMC 



132 



117 



. ^ STUDY PROFILE 

DIMENSION DESCRIPTION CODE 

> 

Unit of Analysis other ^fcymakers) 0 " 01 02 03 04 05 06(07)08 09 

Sampling Purposive 01@03 04 07 08 09 

„ Quasi experimental without r* \ 
Design. . controls and randomization - 01 02 03 04 06 07 08 09 

Research Methods Questionnaire 01 02 03 04 05 @ 07 08 09 

: II 

Analytic Methods Ratings 1 © 3 4 5 8 9 

Analytic Focus Non-qual1tat1ve i 1 (z) 8 9 

Reliability None reported Q 2 8 

i 

1 y 

Validity None- reported 0 2 8 

Definition of Use Instrumental © 2 3 8 9 

Object of Use Uncodable 1 2 3 4 5 6 8 (f) 

Practice Area and 
, Population... Education, policymakers. 
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ASSESSING OPEN-ENDED STATEMENTS OF CONCERN ^ 
ABOUT AN INNOVATION 



AUTHOR: 



Beulah Newlove and Gene Hall 



AVAILABILITY: 



The Research and Development Center for Teacher Education, 
University of Texas at Austin* 



PURPOSE : The purpose of the Instrument 1s to assess the concerns 
of Innovation users and nonusers with regard to the Im- 
plementation of a specific Innovation* Measures arrived at 
are Intended to provide workshop leaders, managers, or 
consultants with a relatively easy and quick method of 
Identifying the concerns of clients or subordinates about 
an Innovation. 



VARIABLES : The Instrument measures the concerns and attitudes of 
respondents. Concerns are defined *s the composite re- 
presentation of the feelings, preoccupations, thoughts, 
or considerations given to a particular Issue. The In- 
strument attempts to measure Individuals' perceptions. 



DESCRIPTION : The Instrument consists pf an open-ended self -administered 

questionnaire which 1s constructed to reflect the specific 

Innovation under consideration. On the first page, 
respondents are asked to respond to the question "Please 
respond 1n terms of your present concerns $\ or how you feel 
about your Involvement or potential Involvement with the 
Innovation of [ ].... Please think 1n terms of your 

own perceptions of what [ ] Involves. Page two 

Includes demographic data concerning the respondent^ length 
of Involvement with the Innovation, their level of awareness 
of 1t, etc. 

The Demographic Information page 1s Intended by the authors 
, to collect "anything you think Important that maybe useful 
1n analyzing the concerns expressed." f^. 

Analysis consists of both qualitative and quantitative data. 
Analysts may read the responses and answer questions such 
as "Are concerns arising from uncertainties about how 
one's role will change with use of the Innovation? Is 
collaboration with other colleagues a predominant focus? 

Alternatively, analysts may break the open-ended replies 
Into "content units" -one or more sentences "representative 



I'M 



.119 



of only one thought or Idea." Each of these content/units 
1s then assigned a number^from the following categorization 
schema : 



0 AWARENESS: 

1 INFORMATIONAL: 



2 PERSONAL: 



3 MANAGEMENT: 



4 CONSEQUENCE: 



5 COLLABORATION: 

6 REFOCUSING: ■ 



Little*, concern about or Involvement 
Innovation 1s Indicated. 



ilth the 



A general awareness of the Innovation and In- 
terest In learning more detail about 1t 1s In- 
dicated; The person seems to be unworried about 
himself /herself 1n relation to th# Innovation. 
She/he is Interested 1s substantive aspects of 
the Innovation 1n a selfless manner such as 
general characteristics, effects, and require- 
ments for use. 

Individual 1s uncertain about the demands of 
the Innovation, his/her Inadequacy to meet 
those demands, and his/her role with the in- 
novation. This includes analysis of his/her 
role in relation to the reward structure of 
the organization, decision making and consider 
atlon.of potential commitment. Financial 
or status Implications of the program for self 
and colleagues may also be reflected. 

Attention is focused on the processes and 
tasks of using the innovation and the best use 
of information and resources. Issues related 
to efficiency, organizing, managing, scheduling, 
and time demands are utmost. 

Attention focuses on impact of the Innovation 
on students in his/her immediate sphere of in- 
fluence. The focus is on relevance of the in- 
novation for students, evaluation of student 
outcomes, including performance and competencies, 
and changes needed to increase student outcomes. 



Trie focus is on coordination and cooperation 
with others regarding use of the Innovation. 

The focus is on exploration of more universal 
benefits from the innovation, including the 
possibility of major changes or replacement 
with a more powerful alternative. Individual 
has definite ideas about alternatives to the 
proposed or existing form of the innovation. 



Original concept from Hall, G.E., Wallace, R.C., Jr., & Dossett, W.A. A 
development conceptualization of the adoption process within educational 
institutions,. Austin: Research and Development Center for Teacher Education, 
The University of Texas j 1973. Taken from "A Manual for Assessing Open-Ended J 
Statements of Concern About An Innovation." Austin: University of Texas 
at Austin, 1976: p. 11./ 
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DEVELOPMENT: 



RELIABILITY/ 
VALIDITY: 



ADMINISTRATION: 



SOURCES: 



The Texas R & D Center developed this procedure out of 
experiences "with Individuals Involved 1n change." The 
process of change brings with It. a number of complex feel- 
ings of uncertainty, doubt and Insecurity rooted 1n 4 indivi- 
duals desire to come to grips with change 1'n the form of 
Innovations being Introduced. The type af concerns indivi- 
duals have vary; "... certain aspects of the Innovation are 
perceived as being more Important than others at a given 
time." At the same time, a desire was felt to understand 
and break down the process of coming to grips with an In- 
novation which Individuals and collectivities normally go 
.through. While the Instrument was first used fo assess 
educational Innovations, 1t has since been used 1n a 
number of other fields. . v - % 



None reported. 




"Description" on preceding page. 



New\ove, Beulah and Gene Hall. A Manual for Assessing 
en-Ended Statements of Concern~^A5out~ Ali Innovation . 
Austin: Research and development Center for Teacher 
Education, University of Texas at Austin, 1976. 

Fuller, F.F. "Concerns of Teachers:' A Developmental 
Conceptualization." American Educational Researc h 
Journal . Vol. 6, number 2, (March 1969) :20775: 

Hall, Gene. "The Study of Individual Teacher and Pro- 
fessional Concerns About Innovations." Journal of 
Teacher Education . Vol. 27, number 1 (1976 J: 22-25. 
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U ' \ ■ ■ 

1 / 

. . STUDY PROFILE 

\ 

DIMENSION : DESCRIPTION , 

- 

Unit of, Analysis . . . , 

* Individual 

Sampling , _ . . 

' Purposive 

Design.... 1 Description £ase Study of 

; Explicit Theory 

Research Methods........! _ ^ 

! Content Analysis 

Analytic Methods....*.... n A . 

Ratings or Codes 

, Analytic Focus „ ' A 1<I(Jb j> . 

. Non Qualitative 

Reliability j 

I Not reported 

V.l*Hiy...:..., Not reported . 

i .4 ' 

Definition of Use 1 - ^ . , 

i Conceptual 

] 

Object of Use | Person-Embodied Innovation 

Practice Area and 

Population Education, Practitioners 



CODE 



@)02 03 04 05 06 07 08 09 
01 (02)03 04 07 08 09 
01 (02)03 04 05 06 07 08 09 10 
01 02(03)04 05 06 07 08 09 



1 

1 (?) 3 4 5 8 9 
l(T)8 9 



0 



2 8 



(f>2 8 
l(?)3 8 9 
1 2 3 4 (T);6 8 9 
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STERN'S ORGANIZATIONAL CLIMATE INDEX: A RECONCEPTUALIZATION 



AUTHOR: 



R.L. Payne and D.C. Pheysey 



AVAILABILITY: 



Payne, R.L. and D.C. Pheysey, "G.G. Stern's Organizational 
Climate Index: A Reconceptualization and Application 
to Business Organizations." Organizational Behavior and 
Human Performance . Vol. 6, number 1, (January i9?i):77^98. 



PURPOSE: 



To design an instrument to measure Business Organizations 
clijriate by reconceptualizing G.G. Stern's Organizational 
Climate Index. 



VARIABLES: The variables chosen by the authors to measure the Business 

Organization Climate Index (BOCI) are: Authority, Restraint, 

• \ work- interest, Personal relations, Routine or control, wider 

k. community orientation administration efficiency. 

(3 0 

DESCRIPTION: The BOCI, questionnaire has 192 items, where twelve to sixty- 
four items combined to form a scale for a single variable 

or factor. Each item is to be scored "True" or "False". 

Typical items include: 

Authority Scales t 

Leaders' Psychological Distance 

Senior personnel are frequently jealous of their authority. 
There is a lot of bootlicking. 

Questioning Authority 

People avoid direct clashes with senior personnel at all 
costs. 

People who. get pushed around here are expected to fight 
back. , 1 

Restraint Scale 

Open.-mindedness 
. People here speak Out openly. 
Criticism is taken as a personal affront in this organization. 



DEVELOPMENT: 



The authors found G.G. Stern's Organizational Climate Index 
(OCI), a rich source of items to measure Business Organization 
Climate Index (BOCI). The 300 items of OCI were sorted into 
groups of items with common interest. By successive sorting 



138 



123 



they arrived at six broad groupings, and forty-six 
subcategories were sorted to their coninon meaning and 
Interest. By this procedure 254 Items were sorted into 
twenty-four different conceptual areas, which" "fit" the 
business organization. The BOCI questionnaire was then 
administered to 12Q junior middle managers" from more than 
100 companies. The respondents were all attending manage- 
ment courses 1n three different Institutions: an Industry 
staff college, a college of technology, and a University. 



RELIABILITY/ 
VALIDITY: 



The 24 subgroups have split-half reliability coefficients 
ranging from .26 to .92 with all but three of the twenty- four 
coefficients greater than .60 and to have Mean general 
blserial correlation. ranging from .68 to .83 (n«120). In 
test-retest reliability eighteen of the scales showed a 
very low F ratio, which supports the reliability of the v * 
Instrument. Only five of the scales had an F ratio much 
greater than 1.0. 



ADMINISTRATION: 



BOCI 1s self -administered questionnaire with 192 Items, 
out of which twelve to sixty- four Items combined to form 
a scale for a single factor. The respondents score "true?- 
or "false" for each Item. BOCI measures use the aggregate 
score of individuals to represent the larger system. 



SOURCES: 



Hickson, D.J., D.S. Pugh and D.C. Pheysey. "Operations 
Technology and Organization Structure: An Empirical 
Reappraisal." Administrative Science Quarterly. 
Vol. 14, (1969)7378^357: * * 

Stern, G.G., M.I. Stein and B.S. Bloom. Methods 1n 

Personality Assessment . Glencoe, Illinois: Free Press, 1956. 

Stern, G.G. People in Context : The Measurement of En- 
vironmental Interaction in SchooT and Society ."""Vol*. 1. 
Unpublished, i»o/. Keport, Syracuse University Library. 

Pheysey, D.C, R.L. Payne.-and D.S. Pugh. "Managerial ' 
Groups 1n Contrasted Organizations: An Analysis of - 
Some Stuctural Concomitants." Administrative Science 
Quarterly . 1971. ' 



ERIC 



l3 f J 



124 



STUDY PROFILE 

DIMENSION DESCRIPTION \ CODE 

ft 

Unit of Analysis.. Organization 01 02 03@05 06 07 08 09 

Sam P l1 "9 - p urposiV e ^1@)3 04 07 08 09 

Cross-sectional with ^-y 

Design... statistical controls 01 02 03(04jD5 06 07 08 09 



Research Methods ... Questionnaire 01 02 03 04 05(06j07 08 09 

Analytic Methods Factor analysis 12 3 (f)s 8 9 

Analytic Focus ..' Multivariate Analysis i 1 8 9 

Reliability 1 

Reported * 

Validity................ Reported 108 

Definition of Use Instrumental Q 2 3 8 9 

Management of business » „ /\ 

Object of Use... ^. organizations environment^ 1 2 (3 )4 5 6 8 9 

• ; ., • — : — . 1 

Practice Area and . 

.Population. .r« Business organization, workers at all levels in the 

business organization 
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CONDUCT AND UTILIZATION, OF RESEARCH IN NURSING 
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AUTHOR: 



AVAILABILITY: 




s 

Donald C. Pelz and doAnne Horsely 



i 

Pelz, D.C. and JA, Horsely./ "Measuring -Utilization of 
Nursing Research" in J.A.Ciiarlo Ced.) Utilizing Evaluation 
Beverly Hills, CA: Sage, ^981. 



PURPOSE: 



VARIABLES: 



DESCRIPTION: 



This questionnaire was developed to assess the nursing staffs 1 
attitudes toward their pontrol over their own practice. 
The general aim of th^/project was to Increase the utili- 
zation of research, findings in the daily practice of 
registered nurses/ 



/ 



Six dimensions of control over nursing practice (COflP) were 
Identified: /Control over practice, access to Ideas, \ 
interpersonal Influence, evaluation and modification, re- 
sources and research utilization. 



The instrument was an eighteen page machine scored question- 
naire. Of the 430 Items of information obtained from the 
questionnaires, ten items and one Index were selected as 
measuring directly or indirectly, the extent of research 
use. i 



DEVELOPMENT : These indices have been used as predictors of staffing 
variables such a$ job satisfaction and have potential 

applicability as predictors of the quality of patient 

care delivered. 



RELIABILITY/ 

VALADITY : Three types of validity and reliability estimates were made. 

Stability. Average percent agreement for the Items in efcch 

of the six indices ranged from 38.0 to 61.0 where missing 
data were replaced by index means. The two-to-three week 
retest period and reactivity of the questions are cited as 
hypotheses for several of the stability estimates falling 
below the 50. OX criterion for new indices. 



Internal Consistency. All six indices met the criterion of 
alpha £.70 for new indices with average alphas for each sample 
« .82. Index factor loadings also met the new Index criterion 
of >.500. The Arizona sample averaged loadings of .679; 
whereas the more heterogeneous Michigan sample averaged .650. 



I4i 



Construct Validity. Although they performed well in dis- 
criminant assessments, Control over Practice and Interpersonal 
Influence did not predict well p<.05 in predictive modeling. 
Therefore, four of these si* new indices are deemed to have 
moderate construct validity, and the other two are still at 
a modest level • 



Thd C.O.N. P. indices were assessed in the context of a cor- 
relational descriptive design with 1 two samples. The first 
was a convenience sample of 116 Arizona registered nurses 
who worked day or evening shift on all major services 
(medicine, surgery, pediatrics, Obstetrics and gynecology). 
The second sample was a random subset of 67 registered nurses 
from a convenience sample of over 1,000 nurses from similar 
inpatient units all over Michigan. The Michigan subset was 
used as a comparison group for the factor analysis and co- 
efficient alpha estimates. Personnel profiles and question- 
naires were used ..for data-gathering. Profiles provided 
information about staffs' education, age, position, previous 
tenure, and sex. 



> 

142 



127 



STUDY PROFILE 

DIMENSION DESCRIPTION CODE 

/ j - 

Ujtft of Analysis.......: Other (nursing staff) 01 02 03 04 05 06@ 08 09 

/ Sampling Convenience (0l)02 03 04 07 08 09 

Design Uncodable 01 02 03 04 05 06 07 OaCjS) 

Research Methods Questionnaire ■ 01 02 03 04 O 5 ^ 7 08 09 

Analytic Methods.. Multivariate 

1 2 305 8 9 

Analytic Focus.... Non-qual1tat1ve 

i 1 &B 9 

Reliability , Reported !(D 8 

Validity Reported \ l(J) 8 

Definition of Use ..-Instrumental (£) z 3 8 9 

Object of Use Uncodable 1 2 3 4 5 6 8 (jT^ 

Practice Area and 

Population Nursing, Practitioners 

i 

/ 1 



. AUTHOR: 



AVAILABILITY: 



PURPOSE: 



VARIABLES: 



DESCRIPTION: 



DEVELOPMENT: 



RELIABILITY/ 
VALIDITY: 
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THE EDUCATIONAL FORCES INVENTORY 



Nicholas Rayder and Bart Body 

Rayder, Nicholas and Bart Body. "The Educational Forces 
Inventory: Psychometric Properties." Journal of Experi- 
mental Education, Vol£4, (1975) :35-44. 



The Educational Forces Inventory (EFI) 1s designed to deter- 
mine the relative Influence of factors within the educational 
system on classroom teaching. 



The following ten variables are used 1n the EFI: 
1) School principle, 2) Central Office Administrative 
Personnel, 3) Other teachers, 4) Parents, 5) District- 
Approved curriculum, 6) Testing programs, 7) Board of 
Education," 8) Physical plant, 9) Social Environment of the 
Community 1n which the school Is located, and 10) The 
teacher him-or herself. 



Each variable listed above 1s evaluated by respondents 1n 
a three-step process (See "Administration" below) 1n the 
form of a self-administered questionnaire which generally 
takes between fifteen and thirty minutes to complete. 
Salient variables are assessed 1n terms of their relative 
Influence on two dimensions: "power" (the amount of -In- 
fluence exerted) and "affect" (4he extent to which the 
Influence 1s perceived to be positive or negative). 



The EFI was developed 1n order to assess the variables 
(or "forces") which classroom teachers believe have the 
.most Impact on the teaching process, either negatively or 
positively! 



Internal validity was assessed by computing correlations 
of randomly paired respondents 1n a study of 700 teachers 
and teaching assistants. Correlations ranged from .47 
to .32. 

Test-retest coefficients ranged from .58 to .65 across 
the three parts of the Instrument with n-il8 1n one school. 
Corresponding coefficients across schools ranged from .80 
to .83 (n-52). 
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ADMINISTRATION ; Each variable ("force") listed 1n"var1ables" above 1s' 
— first ranked on Its perceived Importance on teaching on 

a scale of zero to nine. Next, each 1s assigned a weight 
based on the perceived relative Influence. A total of 100 
points must be distributed among the total number of 
variables. Finally, each variable 1s rated on a positive/ 
negative scale of one to nine, with one being strongly 
negative Influence and nine being strongly positive' In- 
fluence. 

Results are plotted on a two dimensional matrix-power 
and affect. 

S0URCES : Rayder, Nicholas and Bart Body. "The Educational Forces 
- Inventory: A New Technique for Measuring Influences 

on the Classroom." Journal of Experimenta l Education, 
Vol. 44 (&1nter, 1975): 26-34: 
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/ STUDY PROFILE 

DIMENSION DESCRIPTION CODE 

; ^ . 1 

Unit of Analysis individual ® 02 03 04 05 06 07 08 09 



Sampling Random 01 02(03)04 07 08 09 

Quasi -experimental without 
Design control groups or randomization 01 02 03 04^ 06 07 08 09 



Research Methods Questionnaires 01 02 03 04 05(06/17 08 09 

Analytic Methods.* B1var1ate Analysis 1 2 04 5 8 9 

Analytic Focus Non*qual1tat1ve i 1 (5) 8 9 

Reliability.; Reported 1 ® 8 

Validity... Reported 108 

Definition of Use Instrumental Q 2 3 8 9 

Object of Use..' study 103 4 5 6 8 9 

Practice Area and 

Population Education, Practitioners 
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SOCIAL SCIENCE INFORMATION AND PUBLIC POLICY MAKIN6 

it- 



AUTHOR: 



AVAILABILITY: 



Robert F. Rich 

Rich, Robert F.. Social Science Informatjon - and Public 
Pol1cyMak1ng . San Francisco: Jossey Bass Inc., 1981, 



PURPOSE : The purposes of this study were 1) to assess the extent 

to which one of the administrative experiments funded by 

RANN-the Continuous National Survey-was "successful," 2) 
to examine the extent to which the two cultures metaphor 
explains levels of utilization (as opposed to theories 
growing out of studies of organizations and bureaucracies), 
3) to analyze the uses that were made of the Information 
generated through the Contlnous National Survey experiment 
(xv11). . 



VARIABLES :' In analyzing the patterns of utilization, dissemination 

^ — of Information was examined; the kinds and types of. uses 

for the CNS data were assessed (e.g., briefing an assistant 
secretary, writing a memo, etc.); the study examined selective 
utilization of the CNS Information, the form 1n which the 
Information was sent on, what was left out and why. Finally 
the study examined -the perceptions of the respondents con- 
cerning -the most Important factors 1n understanding the 
utilization of Information. 



DESCRIPTION : Four questionnaires were used to conduct the Interviews: 

— one for NORC participants, one for agency participants, one 

for 0MB actors-because of the coordinating role that they 
assumed-and one for the NSF and RANN participants. The 
agency questionnaires covered ten general areas: 1) why 
the agency wanted to become Involved 1n the project; 2) 
how the agency evaluated the success of the CNS experiment; 
3) why the agency decided to continue or discontinue Its 
participation 1n the experiment; 4) the agency's evaluation 
of NORC and the NORC personnel Involved 1n the project; 5) 
how this project related to the agency's other Information- 
gathering and analysis activities; 6) the manner 1n which 
the Information was collected, analyzed, used, and evaluated 
1n comparison to other Information resources; 7) the use 
made of these kinds of data prior to the CNS experiment 
8) the specific use of CNS Information: once the NORC 
report was received, who (within the agency) received 1t 
and what was done with 1t; 9) the utilization of other In- 
formation resources 1n the same policy area to which the 
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CNS Information was being applied; and 10 J the factors • 
critical to the agency's decisions concerning how to use 
Information. In each case, Individual participants were 
asked to distinguish between the agency's evaluation and 
their own Individual evaluations. Participants were also 
asked to Indicate who (Individual or group) was responsible 
for making the agency's evaluation. 

The other questionnaires examined these same general areas, 
emphasizing the particular actor's role. (31-32) 



The model developed by the author calls for the analysis 
of the behaviors and attitudes of the 1) Information 
producers (researchers), 2) Information users (decision 
makers), 3) the sponsors (RANN) and 4) the Office of 
Management and Budget at each of four developmental 
stages of the Knowledge 4 nquiry system (funding, research, 
analysis and reporting of results and summatlve evaluation) 
(28). 



Not reported. 



All actors Involved with the CNS experiment were Interviewed 
and related documentation was collected. The CNS experiment 
lasted twenty- four months— eighteen months of experiment- 
ation and six months after NSF funding expired. During that 
time, thirty-eight Individuals ^ere directly Involved 1n 
providing the Information (NORC), utilizing the Information 
(the participating agencies), and providing the Initial 
funding (NSF and RANN). Each of these participants was 
Interviewed on at least one occasion during the twenty- 
four month period. Twenty-four of these Individuals had 
only minimum contact with the CNS project and were In- 
terviewed on only one or two occasions. The fourteen more 
active participants were Interviewed as long as they con- 
tinued to have contact with the project, on five or six 
occasions. (30-31) 
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\ STUDY PROFILE 


DIMENSION ^ DESCRIPTION 

1 ' ; : 

Unit of Analysis ■ 

S"*""*— Census 

Describe case study with 
Design explicit theory 

Research Methods ........ I nterv i ews 

Analytic Methods Empirical generalizations 

Analytic Focus Non-qual1tat1ve 

Reliability None reported 

Val 1d1 ty . . . . . None reported 

Definition of Use.....*.. . , 

Instrumental 

Object of Use Uncodable 

Practice Area flnd n , . 

Population..... Policymakers and researchers 



^02 03 04 Q6 06 07 08 09 
01 02 03104)07 08 09 



0K02J03 04 05 06 07 08 09 
01 02 03 04(op)6 07 08 09 



•02 3 4 5 8-9 
11(2% 9 
1 (D2 8 " 



02 8 
©2 3 8 9 



1 2 3 4 5 6 8 



® 
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AUTHOR: 



AVAILABILITY: 



Robert R1cfr 



Rich, Robert and Neal Goldsmith. Measuring Knowledge 
Utilization * Princeton; New Jersey: Woodrow Wilson 
School of Public and International Affairs, Princeton 
University, 1981. 



PURPOSE : The purpose of this study is to measure the use of mental 

— 1 health policy research by policymakers 1n the field. 

Specifically public officials at the federal, state and 
local levels conceptualize and "solve" policy problems. 

VARIABLES : The study measured the type pf Information used (e.g., 

service oriented versus finance oriented Information); 

type of use; type of uSer situation or potential areas of 
application; political and organizational constraints. 

DESCRIPTION : The Interview consisted of a number of forced choice and 
* open-ended Items. 



DEVELOPMENT : In this study-; Rich and his colleagues have consciously 
attempted to overcome some of trie limitations of the utili- 
zation field. Most approaches to measuring utilization 
generally ask problem solvers to respond to questions 
concerning specific bits of data within a research-defined 
context. Rich asserts that measuring utilization "out 
of context" results 1n a focus on an -artificial utilization 
process. This study attempts to focus on the overall 
problem-solving situation and not take the utilization 
process "out of cpntext" (41). 



RELIABILITY/ 

VALIDITY: None reported.- 



ADMINISTRATION : An area and posltlonally stratified sample of 479 federal, 

— - state and local level mental health policy makers responded 

to questionnaires and structured personal Interviews. The 
area of application was varied based on service oriented 
versus financial policy needs. 
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• STUDY PROFILE 

DIMENSION DESCRIPTION ' CODE 

• ■ ■ » ,"*•_• 

Unit of Analysis.. Other (mental health policy- : (01 02 03 04 Q5 06©08 09 

makers! 



- Crosseptional without /«v 

Design... statistical controls • 01 02 (jg) 



Sampling "'Purposive 

01003 04 07 08 09 
01 02 (§3)04 05 06 07 08 09 

Research Methods Multiple methods 01 02 03 04 05 06@08 09 

Analytic Methods.. .Multivariate 1 2 3^^5 8 9 

Analytic Focus....- Non-qualitative 'j 1 @8 9 

Reliability -. No estimate reported ^^2 8 

Validity : _ (l)z 8 



No estimate reported. 



© 



Definition of Use T , . (1 )2 3 8 9 

Instrumental 

Object of Use... " . 1 2 3 4 5,(98 9 

% Other « K ~ y 

Practice Area and, r . . • _ ' t . " , . ' 

Population Federal, State and Local mental health, policymakers 

3 
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DEPTH OF UTILIZATION SCALE 

\ ' 

v 

i 

Jack Rothman 



Rothman, J. f et al. Diffusing Human Service Innovations : 
A Social Marketing Approach . Beverly Hill, CA: Sage, 
Tn press. 



The purpose of this scale is to-measure the degree to which 
a knowledge derived product carried through development 
and put in user- ready form is actually utilized by intended 
users. In this study, the authors focused on the behavioral 
aspects of the use of, a handbook. The Depth of Utilization 
Scale was used as the primary dependent variable for 
appraising diffusion results, v r ■/ 



The scale measures behavioral aspects of util ization. The 
questionnaire solicited Information concerning handbook 
usage and general reactions, factors presenting obstacles 
to guideline implementation and general practitioner 
demographic data. 



The instrument M was a one-page two-sided instrument. The 
questionnaire-attempted to assess the depth of utilization 
of the handbook, asking a series of questions that probed 
small, incremental degrees of use. This ranged from failure 
to receive or examine the handbook to using it t<j implement 
a guideline fully with a high degree of goal attainment. 
The following are examples of some of the items used in 
the Depth of Utilization Scale: 

* I did not reqeive a handbook. 

* After examining the handbook I seriously considered 
applying it to my practice. 

* I partially implemented a specific action guideline. 

From the questionnaire data, the authors used a combination 
of a contingency table analysis and a descriptive ratio, 
which they refer to as the "Depth of Utilization Ratio" (DU£) 
The OUR is a measure of the extent tfo which handbook material 
was actually used by practitioners. Its more formal defin- 
ition is the "aggregate utilization score for all respondents 
exposed to a tjiven diffusion approach relative to the 
maximum possible aggregate score for that subsample of 
respondents." The formula used to calculate this ratio was: 
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Aggregate Depth of Utilization Score 

Number of Respondents x 3 (max. utilization score) 



= DUR 



This ratio facilitates a descriptive comparison of the 
amount of utilization resulting from each diffusion approach. 
(3-24, 25) 



DEVELOPMENT : The questionnaire used to assess the "Depth of Utilization" 

was a shortened, modified version of an instrument used 

in another Social marketing experiment. From this original 
questionnaire, a nine-point scale was developed for r^ing 
degrees to which practitioners used the handbook. 



RELIABILITY/ 

VALIDITY : Not reported. 



ADMINISTRATION : Those individuals who were listed on the agency order cards 

to receive handbooks (2688 in all) became the sample for 

^flalyzing utilization . In order to determine the degree 
of utilization, each practitioner was sent the questionnaire 
after a three-month implementation period. We requested 
him or her to return the questionnaire directly to our 
office. Included with the questionnaire was a brochure 
reiterating the specific appeal assigned to that practi- 
tioner's agency and a letter describing the questionnaire's 
purpose. (3-23) 
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STUDY PROFILE 

DIMENSION DESCRIPTION , CODE 

ft 

Unit of Analysis Individual @K.03 04 05 06 07 08 09 

Sampling. Census ' 01 02 03 @07 08 09 

Design. Uncodable ( , 01 02 03 04 05 06 07 08(S) - 

Research Methods Questionnaire " 01 02 03 M 05(66^07 08 09 

Analytic Methods Blvari ate Analysis 1 2(5)4 5 8 9 

. 4 ' 

Analytic Focus -Non-Qualitative ' * 9 

Reliability , Not reported (£)* 8 

t 

Validity (T)2 8 

Not reported 

Definition of Use Instrumental fi) 2 3.89 

Object of Use Uncodable 1 2 3 4 5 6 a(sl) 

• j ! ' 

Practice Area and 

Population Human Service Delivery, Practitioners 
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AUTHOR: 



AVAILABILITY; 



PURPOSE: 



VARIABLES: 



DESCRIPTION: ' 



DEVELOPMENT: 



USING RESEARCH IN ORGANIZATIONS / 
Jack Rothman 

Rothman, J. Using Research . in Organizations '. Beverly Hills, 
CA: Sage, 1980. ■* * . 



The purpose of this study was to explore ways that the » 
linkages between research and its application could be 
enhanced. The study examined ways of structuring and 
carrying out the research function so as to maximize the 
probability of research findings being used by organizations 
in their operations. 



The variables the author examined can be. broadly classified 
Into four groups: 1) structural factors (e.g., top-level 
support 1n the organizational system, structural access 
by researchers to applied personnel 1n planning and service 
delivery functions); 2) the research process; 3) organiza- 
tional climate (e.g., the social psychological climate of 
attitude and relationship within an organization); 4) reports 
and products of research (e.g., form Into which the results 
of research are placed). 



The author employed a structured questionnaire that contained 
mostly open-ended items. The procedure was flexible, 
allowing the respondent to develop his thoughts at length 
and to move freely from Idea to idea. The majority of the 
Interviews averaged over two hours. 



During the -interview the respondents were asked to recall an 
Instance 1a which a specific research study was used as a 
basis for a planning decision within the aSgency. Thefy were 
then urged to trace the Influences involved 1n such a success- 
ful case pf utilization. 

They were asked on a more general level, questions such as, 
"Do you feel there are factors that work in favor of, or 
against, the use of social research Information 1n your 
organization? What are the factors that work for the use 
of such information? (25) 

This study was not designed to investigate aga^n the 
frequently asked question of whether research is or 1s 
not used in organization operations. The author assumed 
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some research is used and his main questions were related 
to determining the conditions that are associated with 
greater utilization. 



RELIABILITY/ 

VALIDITY : Not reported. 



ADMINISTRATION : The study focused on the use of information in planning 

— * decisions. The study specifically concentrated on instances 

where research information had been successfully applied. 
Interviews were conducted in twelve local departments around 
London. The twelve departments were carefully selected 
using a panel of knowledgeable^ experts. Both the director 
and the senior research person were interviewed. In all, 
twenty-four informants participated. 
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DIMENSION 



STUDY PROFILE 



DESCRIPTION 



Unit of Analysis. 

Sampling 

Design 

Research Methods. 
Analytic Methods. 
Analytic Focus... 
Reliability 



Organization 



Purposive 



Not codable 



Interview 



Ratings 



Not qualitative 



Not reported 



Validity. 



Not reported 



Definition of Use. 



Instrumental 



Object of Use. 



Uncodable 



Practice Area and 
Population 



CODE 



(jT)) 



01 02 03 (04 JD5 06 07 08 09 



01 (52J03 04 07 08 09 
01 02 03 04 05 06 07 08(09 
01 02 03 04(05)36 07 08 09 
(l) 2 .3 4 5 8 9 
1 108 9 
0-2 8 



Ql 8 
(P) 2 3 8 9 
1 2 3 4 5 6 8 (7) 



Social services planning, Researchers and administrators 
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LOG REPORTING FORM 



Jack Rothman, Joseph G. Teresa and John Erllch 



Rothman^ J. Teresa, J.G. and J. Erllch. Developing Effective 
Strategies for Social Intervention: A Research and 
^ Development Methodology . Springfield, VA: NatlonaTTTechnical 
Information Service, 1977. 



The main purpose of the Log Reporting Form is to record 
the process of implementation of a knowledge-derived appli- 
cation concept (practice guideline) and to assess the 
success of guideline implementation. 



The criteria for evaluating success of guideline implement- 
ation were: 1) Innovation: the degree to which innovation 
was carried out in or 'accepted by proximate general target 
system; 2) Role Effectiveness: degree to which intended 
effectiveness outcome was attained by exercising the role; 
3) Changing Organizational Goals: the degree to which an 
organization's goals were changed; 4) Fostering Comnunlty 
Participation: the degree to which participation 1n target 
group was increased/modified. (123) 



As a result of the authors' early pilot work, three types 
of logs were clearly necessary in order to record the entire 
process of implementation. These were the Initial Log" (IL), 
a set of Periodic Logs (PL) and a Final Summary Log (FSL). 

The Logs" had the function of capturing essential facets of 
implementation of a practice guideline. Structural response 
categories were used to offer a means of comparative 
analysis across different "cases" of Implementation and 
operationalization. In addition there were open-ended 
narrative items in the log designed to encourage field 
staff to note 1n<the1r own words any additional relevant 
aspects. 



the log recording instrument evolved in stages through 
numerous pilot examinations and modifications. The log 
was conceived as an instrument for recording the process 
of operationallzation and for measuring guideline outcome. 
A preliminary draft log was used 1n a prestudy with eight 
practitioners. 



RELIABILITY/ 

VALIDITY : , Not reported. 



ADMINISTRATION : The purpose of the project was to carry out a. field test of 

selected action principles concerning intervention at the. 

community and organizational level. The intent of , the 
field test was to quantify the process of : implementation 
of the guidelines in the field and to develop a procedural 
manual to facilitate successful application of the action 
guidelines. 
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STUDY PROFILE 

DIMENSION DESCRIPTION CODE 

Unit of Analysis Individua1 @02 03 04 05 06 07 08 09 

1 T 1 | - 

San * l1n9 ' - Convenience . @02 03 04 07 08 09 

Design...,...: Uncodable 01 02 03 04 05 06 07 08 (S) 

Research Methods Content Analysis / 01 02 ®° 4 05 06 07 08 09 

Analytic Methods Empirical Generalization 

(T)Z 3 4 5 8 9 

' y : '- 

Analytic Focus Qualitative (J) 2 89 

Reliability No estimate reported CD 2 8 

Validity.". --^©-estimate reported @2 8 

Definition of Use Instrumental (J)2 3 8 9 

Object of Use.... Uncodable 1 2 3 4 5 6 *Q) 

Practice Area and 

Population Human Service Delivery, Practitioners 

* 

» 
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PLANNING AND ORGANIZING FOR SOCIAL CHANGE 
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AUTHOR: 



Jack Rothman, Joseph G. Teresa and John Erlich 



AVAILABILITY: 



Rothman, J. f J.G. Teresa and J. Erlich. Planning and 
Organizing for Social Change , Columbia UniversityTress, 
1974. 



PURPOSE : The purpose of .this study was to develop generalizations and 
action guidelines for planning and implementing social change, 

VARIABLES : The Study Report Form measured several characteristics of, 

* studies including, the variables investigated, the design 

of the study and the findings. . 



DESCRIPTION : The main instrument of the investigation was the Study 
Report Form on which was recorded information for each 

study included in the data pool. The instrument has three 

main sections: 

* 1. Background information concerning the study . This is a v 

kind of "vital statistics" overview: the variables studied 
or themes covered; the study design and methodology; the 
national and community context in which the study was 
conducted. 

t. An abstract or summary of the study . This included: a) 
elaboration on the theoretical perspective or problem 
investigated Including conceptual framework or hypotheses; 
b) a reasonably well developed description of the method- 
ology; c) a statement of the major findings. 

3. Coding of data into practice issue areas (such as part- 
icipation, organizational behavior, practitioners' roles , 



etc . 
on 



.) and drawing policy or practice Implications based 
the data. Here the reviewer coded the study within 



all categories Into which it fell and was urged to 
consider matters of application and utilization for each 
of these categories^ 



DEVELOPMENT: 



Not reported. 



RELIABILITY/ 
VALIDITY: 



Not reported. 



9 
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ADMINISTRATION : There were several steps Involved 1n categorizing and coding 

— the data. First, they went directly into the data and 

attempted to discern Inductively the kinds of categories 
IntOHWhich material- naturally fell. Subsequently, they 
surveyed the literature of that area in order to determine 
the topology of the field as developed theoretically by 
scholars who had written in the area. In some instances 
where there was a strong established theoretical school 
they leaned in the direction of devising subcategories 
(as with organizational theory and the diffusion of in- 
novations). In other Instances, where no commonly accepted 
overarching framework existed among scholars, they leaned 
in the direction of composing their own subcategories (as 
" with practitioner roles and participation). 

The next stage was grouping elements, and making connections 
between different languages, concepts, and findings from 
diverse disciplines and contexts. Clusters of data com- 
prising consensus findings were constructed and appropriate 
statements constituting generalizations were composed. 
Finally Action Guidelines were drawn, based in part on 
application derivations suggested in the report forms for 
each of the studies. (584-85) 
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DIMENSION 



STUDY PROFILE 



DESCRIPTION 



CODE 



Unit of Analysis. 
Sampling v 



Design * .', 



Research Methods. 
Analytic Methods. 
Analytic Focus. . , 



Uncodable 



01 02 03 04 05 06 07 



08(1)9) 



Purposive 



Uncodable 



01(02)03 04 07 08 09 



01 02 03 04 05 06 07 08(09 




Content Analysis 



Reliability Not reported 



01 02(03^04 05 06 07 08 09 

Empirical Generalizations * (7) 2 3 4 5 8 9 

1(2)8 9 
2 8 



Non-qualitative 



Validity.^ '. Not reported 



Definition of Use Instrumental 

Object of Use 



"0 



2 8 

2 3 0 9 



Study 



1(^T)3 4 5 6 8 9 



Practice Area and 
Population. 



Human Service Delivery, Studies 
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INTERPERSONAL TRUST SCALE 



AUTHOR: 



Julian Rotter 



AVAILABILITY: 



PURPOSED 



Rotter, Julian. "A New Scale for the Measurement of 
Interpersonal Trust," Journal of Personality Vol. 35, 
number 4, (December 1967) : 651-6?. 



The I.T.S. was developed to create an index showing 
relative levels pf trust individuals hold towards 
"social objects"«individuals and social institutions 
(judiciary, public officials, schools, and authority 
figures.) 



VARIABLES : The variable consideration is the level of trust as 

measured by an Interpersonal Trust Score. (See 

"Administration" below). 



DESCRIPTION : The scale consists of thirty five statements. 



Examples: "Most people can be counted on to do what 
they say they will do." 

"There is no simple way of deciding who 1s 
telling the truth." 

> "The judiciary is a place where we can all 

get unbiased treatments." 

Respondents score each statement with a value from one 
(strongly agree) to 5 (strongly disagree). 



DEVELOPMENT: 



Not reported. 



RELIABILITY/ 
VALIDITY: 



Seven month test-retest reliability was .56(n«24). 
Validity correlations with "observations of everyday 
behavior by sociometric techniques" was reported to 
be "good" for both construct and discriminant validity. 



ADMINISTRATION: 



The ITS is a self -administered questionnaire. Scores 
are obtained by summing scores from each Item. Higher 
scores represent higher levels of interpersonal trust. 
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SOURCES ; Not reported. 




PROBLEM SOLVING IN NURSING 



Joyce Ringstad Schmidt 

< 

Schmidt, Joyce Ringstad, Problem Solving in Nursing 5531 
Virginia, Clarendon Hills, IL 60514 



The instrument was designed to measure problem-solving 
ability in areas typical ot a nurse's experience. 

Probl em-solving ability is defined as the ability to 
investigate problems in a systematic way in order to 
resolve them; the process involves the following 
variables: recognizing and defining the problem, 
collecting and analyzing related data, formulating possible 
solutions, and choosing a course of action which is. then 
tested and evaluated* 



Six nursing care problems are described, one per page. 
Working on one problem at a time within seven minutes, 
respondents write as many solutions as they can think of 
for that problem. Four additional questions ask respondents 
to indicate for feach problem: 1) which solution they 
think most beneficial 2) whether the respondent would 
actually try the selected solution in a hospital; 3) 
whether the selected solution would be used* by most 
nurses; 4) whether solution selected is already prac^ J 
ticed at the hospital. Responses are scored by counting 
the number of solutions provided for each problem. Sample 
items of the problems are: 

A- Mrs. Arthur, a 55-year-old grandmother who loves gardening, 
is in a private room. She has had neck surgery and is 
not allowed to move her head from side to side. She maybe 
in Fowler's position. She faces the wall and complains * 
of boredom. 

B- Three-year-old Johnny is on a general diet. He will 
eat any type of food but dislikes drinking liquids. He 
is to have 1,000 cc in fluids per day. 



The instrument was pretested with six senior diploma* school 
nursing students who were chosen by two faculty as represent- 
atives of high, medium, and low academic and clinical 
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standing 1n the school. Suggestions regarding clarity and 
format were solicited and the amount of time needed to 
respond to items was noted. 

Three nursing educators evaluated the problems with regard 
to whether they were more or less typical of the\kinds of 
problems faced by nurses; whether creative thinking could 
be applied in solving the problems; whether students other 
than freshmen would be capable of solving them; and whether 
the probT&n was relevant to nursing. On the basis of these 
evaluations, six problems were retained for the final version 
of the instrument. 



RELIABILITY/ 
VALIDITY: 



A test-retest coefficient (one-week interval) was calcu- 
lated using ten associate decree college nursing sophomores 
nearing completion of their nursing program. Using the 
number of solutions given as a criterion measure, test- 
retest coefficients ranged from .50 to .92 for the various 
problems, with a value of .96 for the total test. 

Some aspects of content validity were addressed by the pro- 
cedure used. The instrument, along with the Torrance test 
of creative thinking (verbal Form A), was administered to 
thiry-six juniors and twenty-eight senior students from 
one diploma program. The correlation between the total 
creative thinking score and problem-solving was r=.43 (p .05), 



ADMINISTRATION: 



Administered in groups (from one to 8 correspondents) with a 
proctor. Seven minutes are allowed for each problem with 
a warnirtg given at one minute before the end of each 
seven-minute period. There are two parts to the instrument. 
The first part with problems asks for the solutions and the 
second part refers. to the question about the suggested 
solutions. 



SOURCES : Torrance, E. Paul. Torrance Tests of Creative Thinking : 
i — Technical Manual . Princeton: Personnel Press, 1966. 

Ringstad, Joyce Laura. Creative Thinking and Problem 
Solving Abilities in Nursing Students . Unpublished 
master's thesis, University of Illinois Medical 
Center, 1972. 
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STUDY PROFILE 



DIMENSION [ 

f 

Unit of Analyses 

Sampling 

Design - 

Research Methods \ 

Analytic Methods 

Analytic Focus 

Reliability j 

i 

j 

Validity.... j 

Definition of Use 

Object of Use 

j 

Practice Area and 
Population 



DESCRIPTION 



CODE 



Individual 



Purposive 



Not Available 



Questionnaire 



Uncodable 



Qualitative 



Reported 



Reported 



Instrumental s 



Uncodable 



,6l)o2 03 04 05 06 07 08 09 



01VQ2 03 04 07 08 09 



01 02 03 04 05 06 07 ^ 09 10 



01 02 03 04 05@ 07 08 09 

1 2 3 4.5 8(9} 



I) 2 8 9 
1 (D 8 

1 ©8 

(T) 2 3 8 9 



Nursing Education, Nurses 



|?1 2 3 4 5 6 8(9) 
4 
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CONCEPTIONS OF RESEARCH QUALITY 
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AUTHOR: 



Andrew Seldel 



AVAILABILITY: 



Seidel, A.D. "Underutilized Research," Knowledge : Creation , 
Diffusion , Utilization . Vol. 3, Number 2 (December. 1981): 
233-248. " 



PURPOSE : This study was designed to address the question of whether 
research products are underutilized because the researchers 

who produce the information and decisionmakers who are paten- 
is, tial users of the information do not share common conceptions 

of information quality. 

VARIABLES : This study measured perceived research quality, the relatlon- 

ship between perceived usefulness and relevance and Incentive 

systems. v r 



DESCRIPTION: 



Respondents were asked to rate a 11st of qualities or criteria 
that one might use to evaluate environment and behavior re- 
search On the following scale; 

1 - greatly detracts from Information quality 

2 - detracts from Information quality 

3 - no significant effect on Information quality 

4 - Increases Information quality 

5 -greatly Increases Information quality 

6 - No rating possible - affect 1s too dependent on situation 

7 - Don't know 



DEVELOPMENT: 



Not reported. 



RELIABILITY/ 
VALIDITY: 



None reported. 



ADMINISTRATION: 



The study consisted of a personal Interview which lasted 
approximately two hours. The Interview was followed by a 
questionnaire that was approximately one-third structured 
questions and two-thirds open-ended, focused interview 
questions. 
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\ STUDY PROFILE 
DIMENSION : DESCRIPTION CODE 

* - r 

Unit of Analysis. Individual (oi) 02 03 04 05 06 07 08 09 

Sampling Purposive ' 01 ©03. 04 07 08" 09 

_ , Crossectlonal without n , M rr\ n . „_ rt _ rt _ 

Design statistical controls 01 02 ® 04 05 06 07 08 09 

^Research Methods........ Multiple methods 01 02 03 04 05 06@ 08 09 



Analytic Methods Ratings or Codes 

1 © 3 4 5 8 9 

Analytic Focus •„« Qualitative 

i© 2 8 9 

Reliability None reported © 2 8 

Validity... None reported © 2 8 

Definition of Use Conceptual . 1 ® 3 8 9 

Object of Use Uncodable 1 2 3 4 5 6 8 ® 

Practice Area and ^ 

Population. Architects (practitioners) anc^Eiwa^5^ent and 

Behavior researchers 



i 7u 
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HALL'S PROFESSIONALISM SCALED AN 
EMPIRICAL REASSESSMENT 



William E. Snizek. 



Snizek, William E?. "Hall's Professionalism Scale: An. 
Empirical Reassessment". American Sociological Review 
Vol . 37 f (February 1972) : 109-1 14. 



Hall's Professionalism Scale was developed to measure 

the degree of "professionalism". It is a generic scale, 

applicable to any self-identified profession or group 

of workers. Snizek* s Study sought to replicate the original 

study, and in so doing to test the reliability^ the 

original instrument and to suggest modifications of the 

survey. 



The concept of "Professionalism" was broken down by Hall 
into five "attitudlnal components": l)Using a professional 
organization as a major reference point in job related 
activities; 2) a belief that the profession is "indisposable 
and beneficial" to society; 3) a belief in self-regulation; 
4) a definition of work being an end in itself and not a 
means to an end an$l 5) Autonomy— the degree to which the 
practitioner considers him or herself able to make decisions 
about the job. 



The Professionalism scale originally used by Hall contained 
fifty items, ten for each of the five components of 
professionalism. The original study was based on 328 subjects 
from eleven occupational groups: physicians, nurses, 
accountants, teachers, lawyers, personnel managers and 
advertising executives. 

For a description of Snizek's survey see "Administration" 
below. 



Both HalTs original survey and Snizek's reassessment are 
based on the desire to operationallze the term "profession- 
alism". The similar attitudes and ideology" held by in- 
dividuals undertaking similar tasks and confronting similar 
Issues are, taken together, what constitute a "profession;" 
the deliberate or unconscious alteration of characteristics 
in work-style and attitudes constitute "professionalizatlon." 
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RELIABILITY/ 

VALIDITY ; Stratified Reliability Coefficients for the Hall versus 

Snizek data (reproduced in the table below) suggest 

reducing the number of items per dimension from -ten to 
five. Reliability coefficients for all dimensions does 
) not drop significantly as a result. 



STRATIFIED RELIABILITY COEFFICIENTS: 
Hall vs Snizek 



Dimension 


Hall 


Data 


Snizek 


Data 




HO items 


5 items 


10 Items 


5 items 


1 


0.674 


0.686 


0.620 


0.621 


2 


0.676 


0.742 


0.656 


0.640 


3 


0.694 


0.711 


0.596 


0.699 


4 


0.711 


0.703 


0.455 


0.583 


5 


0.776 


0.760 


0.730 


0.738 


All 










Dimensions 

m 


0.860 


0.843 


0.799 


0.783 



Source: Snizek, William. "Hall's Professionalism Scale: An 

Empirical Reassessment, 11 American Sociological Review , 
Vol. 37, (February, 1972)111?: 



ADMINISTRATION : Snizek administered the professionalism scale instrument 

to 566 aeronautical, nuclear and chemica/1 engineers, 

personnel managers and chemists. In both surveys,- the 
level of education of respondents varied: B.A. or B.S., 
M.A. or M.S., PhD. and M.D. Utilizing both sets of data, 
Snizek used Orthogonal Varimax factor analysis to determine - 
the decree of fit of the items used to measure each of the 
five components in the scale. Results (See "reliability/ 
validity" above) indicated an "unacceptably low factor loadirrg" 
on their intended theoretical dimension. Several items show 
such low loading on all of the five dimensions. Other items 
showed correspondence with more than one dimension. 



SOURCES : Hall, Richard. "Professionalization and Bureaucratization." 

American Sociological Review , Vol. 33 (February 1968): 

92-104. 

— * Occupations 'and the Social Structure . Engl wood 

CI i f f s : Prentice-Hall7T9597 

Vol lmer, -Howard M. and Donald Mills (eds.). Professionalization . 
EngTewood Cliffs: Prentice-Hall ,1966. 
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" STUDY PROFILE 

DIMENSION DESCRIPTION CODE 

ft 

Unit of Analysis other (professionals) 01 02 03 04 05 06@08 09 

Sampling Purposive Ol(02)o3 04 07 08 09 

Quasi -experimental without S\ 

Design control groups and/or 01 02 03 04(05J06 07 08 09 

randomization 

Research Methods... Questionnaires 01 02 03 04 05(0^07 08 09 

Analytic Methods. * Multivariate Analysis 1 2 3(7) 5 8 9 . 

Analytic Focus Non-Qualitative ■ i 1 (?) 8 9 > 

Reliability Reported (T)2 8 

Validity.... Reported . 8 

Definition of Use Instrumental (jj2 3 8 9 

-Object of Use Person-embodied 1 2 3 ,4(s)6 8 9 

Practice Area and ... 
Population : General 

-j 

* 
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TAXONOMY OF ETHNOGRAPHIC QUESTIONS 



James Spradley 



Spradley, James. The Ethnographic Interview , New York: Holt, 
Rinehart, and Winston, 1979. 



Ethnography-"The work of describing a culture"-is fully 
described in both its theoretical underpinnings as well 
as methodology and technique. 



"Instead of collecting 'data 1 about people, the ethnographer 
seeks to learn from people, to be taught from." This quote 
captures the essence of the ethnographic method: it is an 
attempt to discover cultural and social characteristics 
of an individual or collectively from the viewpoint of the 
subject. 



In the process of ethnographic research (See "Administration" 
below), a series of "ethnographic questions" or issues are 
considered. A summary of these issues is below. It is . 
important to note that the author insists that these 
questions are indivisible, in the sense that each step 
in the ethnographic process depends on the" successful 
completion of the previous step. For a detailed description 
of this series of issues, see Spradley above. 

A Taxonomy of Ethnographic Questions 

1.0 DESCRIPTIVE QUESTIONS 

1.1 Gra^ Tour Questions 

1.11 Typical Grand Tour Questions 

1.12 Specific Grand Tour Questions 

1.13 Guided Grand Tour Questions 

1.14 Task-Related Grand Tour Questions 

1.2 Mini-Tour Questions 

1.21 Typical MinirTour Questions 

1.22 Specific Mini-Tour Questions 

1.23 Guided Mini -Tour Questions 

1.24 Rask-Related Mtni-Tour Questions 

1.3 Example Questions 

1.4 Experience Questions 

1.5 Native-Language Questions 

1.51 Direct Language Questions 

1.52 Hypothetical -Interaction Questions 

1.53 Typical -Sentence Questions 
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2.0 STRUCTURAL QUESTIONS 

2.1 Verification Questions 

. 2.11 Domain Verification Questions 

2.12 Included Term Verification Questions 

2.13 Semantic Relationship Verification Questions 

2.14 Native-Language Verification Questions 

2.2 Cover Term Questions 

2.3 Included Term Questions 

2.4 Substitution Frame Questions 

2.5 Card Sorting Structural Questions 

3.0 CONTRAST QUESTIONS 

3.1 Contrast Verification Questions 
3.2\Directed Contrast Questions 

3.3 Dyadic Contrast Questions 

3.4 Trladic Contrast Questions 

3.5 Contrast Set Sorting Questions 

3.6 Twenty Questions Game ^ 

3.7 Rating Questions 



DEVELOPMENT : Ethnography rests on a number of critical assumptions 

— — — concerning the relationship between research and the phenom 

ena which 1t 'attempts to study, the social nature of 
research contexts, and the failure of much "data oriented" 
research to capture fully the cognitive maps or frames 
of reference of the subjects culture- a 'set of principles 1 
by which meaning is Inputted to social and physical events, 
action is guided and sanctioned and domains of social life 
- become meaningfully Interrelated. 



RELIABILITY/ ^ 

VALIDITY : Reliability atid validity estimates are not applicable to 
■ «- this study. 

ADMINISTRATION : Writing the final ethnography is thfe product of a multiple 

step process of writing briefer interim reports, each of 

which focuses on one aspect of the research process. A 
summary of the twelve steps suggested by Spradley, are 
reproduced b61ow. 

Developmental Research Sequence 
Writing Tasks 

(1) Locating an Informant 

»* (2) Interviewing the Informant 

(3) Making an Ethnographic record 

- (4) Asking Descriptive Questions 



9 
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(5) Analyzing Ethnographic Interviews 

(6) Make" a Domairi Analysis 

(7) Asking Structural Questions 

(8) Making a Taxonomic Analysis 
{9) Asking Contrast Questions 

(10) Making a Componentlal Analysis 

(11) Discovering Cultural Themes 

(12) Writing the Final Ethnography 
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ORGANIZATIONAL BARRIERS JO INNOVATION 



AUTHOR: 



L. Glen Strasburg r 



AVAILABILITY; 



NA$A Research Paper number 23, Graduate School of Business, 
University of California, Los Angeles. 



PURPOSE : The procedure has as its primary purpose the explication 

and determination of variables pre3£pt 1n organizations 

which serve as potential Incentives or barriers to the 
origination, development and implementation of innovations. 



VARIABLES: 



Variables are "organizationally based" barriers to Innovations, 



DESCRIPTION: 



Responses froto self administered questionnaires are coded 
by researchers to jdentify categories or classes of incentives 
or barriers to innovations within the organizations. Ma- 
trices are constructed which indicate both the nature of the, 
barrier as well as the level of the barrier~"Super1or" 
(barrier stemming from above), "Management" (barrier stemming 
from the management system), "Peers" (barrier stemming from 
fellow employees), or "Self" (barrier stemming from perceptions 
of the individual ). 

Seventeen categories of barriers were identified, ranging 
f font technical ("Physical Facilities,"' "Equipment") to* 
conceptual "Ability to Evaluate," "Responsiveness to Ideas." 



DEVELOPMENT: 



N/A 



RELIABILITY/ 
VALIDITY: 



No estimates given; although "consensus" among coders was 
expected as to the categorization scheme. Finally 
agreed upon, no specific information was given. 



ADMINISTRATION: 



A self -administered questionnaire is coded by members of 
the research team. Several iterations of both individual 
and group coding are undertaken. 
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STUDY PROFILE 

k 

• » <f 

DIMENSION ' DESCRIPTION CODE 

Unit of Analysis Ratings Ol(o2)03 04 05 06 07 08 0? 

Sampl1n <> -— Random-like 01 A2@04 07 08 09 

araf 01 ° 2 ®° 4 05 ° 6 07 ° a «>< 

Research Methods.. - Questionnaires 01 02 03 04 05 ©07 08 09 

* Analytic Methods 1 2 3 4 5 8 9 

Analytic Focus Non-qualitative , + - > 1 © 8 9 

— . — : — 

Reliability Not reported Q) 2 8 

Valldlty..^^ Not reported ® 2 8 

Definition of Use .-Conceptual *1 © 3 8 9 

Object of Use Uncodable 1 2 3 4 5 6 (S) 9 

Practice Area and 

Population Engineers, government laboratory 

\ 

\ 

o 

o « 17b 
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•PROFESSIONALIZATION SCALES' W~ 



AUTHOR: 



Madeleine C. Vaillot 



AVAILABILITY: 



PURPOSE: 




Sr. Madeleine Clemence Vaillot, R.N., Dominican Sisters 

of the Presentation, 3012 Elm Street, Dighton, Massachusetts 

02715. J 

The professlonalization scale attempts to break down into 
Its component parts "professional orientation 11 on the part 
of nurses. 



VARIABLES : t Professionalism of Nursing, Self perception.^rceptions 
? ' of physicians and patients concerning nurses^ffrid nursing, 



DESCRIPTION: 



The scales consist of twenty-five items, each of which 
requires six answers concerning what the respondent 
perceives: 1) ought to be true in nursing, 2) teachers/ 
in schools of nursing would consider to be ideal, 3) 
physicians think ought to- be true, 3) head nurses think 
ought to be true, 5) patients think .ought to be true, and 
6) is true in nursing. Each of the twenty-five i tems is 
followed by three statements that reflect different degrees 
of professional ization; one statement is selected for each 
of the six contexts listed above. ■ ' 



DEVELOPMENT: 



RELIABILITY/ 
VALIDITY: 



In order to construct the thirty rating-scale item survey, 
nursing literature was examined to locate statements char- 
acteristics of professional orientation on the part 'of 
nurses. Several implicit hypotheses form the rationale for 
questionnaire Items: 1) the attitude among, nurses that 
nursing activities are finds in themselves; 2) nurses 
protect themselves from encroachment by those persons they 
perceive to be lower in the health care hierarchy; 3) 
higher education is a means to professional status; 4) 
nurses resent attitudes of physicians towards nurses and 
nursing; 5) nurses should set tlje standards of nursing 
education, and 6) nurses are responsible only to A the 
nursing! profession. , < 



No data on reliability were provided. 

Content validity was addressed by having nurse educators 
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review the items during development of the tool. Stems of 
Items were examined to see„i,f they accorded with their 
intended objectives (Vaillot, 1962). 

Vail lot (1962) reported a study relevant to construct 
validity. Subjects were 116 *freshmen and seventy-six 
seniors at three diploma schools of nursing and twenty-nine 
senior students at a school :for practical nurses. Analysis 
of the professional school iata revealed no significant 
differences among 4 schools. Significant differences between 
scale scores were found, with teachers perceived as the 
most professional group and patients as the least. Scale 
' scores for the practical nursing students tended to be 
ordered in the same way as scores, for diploma-students. 

ADMINISTRATION : The survey is composed of twenty-five rating-scale items. ' 

The survey is self-administered, and takes between fifteen 
and twenty minutes to complete. Six scale scores result 
from analysis. Scale scores are obtained by summing the 
values assigned to choices made by the respondent (0, 1, or 2) 
Summed scores for each scale range from zero to. fifty. 

" Cm 

S0URCES : WArcher, M.L.B. Establishing Equivalency for Nurses Seeking 
Admission to Graduate Nursing Programs . Unpublished 
doctoral dissertation, Uni versi ty of Mi ssouri— Kansas 
City, 1976. 

, , Richards, Mary A.B. "A Study of Differences in Psychological 

Characteristics of Students Graduating from Three Types 
of Basic Nursing Programs." Nursing Research. Vol* 21, 
(1972): 258-261. 

Vaillot, M.C. Commitment to Nursing : A, Philosophic Invest- 
igation . ' Philadelphia: Lippincott/ 1962. 
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STUDY, PROFILE 

DIMENSION DESCRIPTION . ■ ' CODE 

ft 

Unit of Analysis other (nurses) 01 02 03 04 05 06 ©08 09 

Sampling. Purposive 01(02)03 04 07 08 09 

Quasi -experimental without 

Design...;.. control groups and/or random- 01 02 03 04/O5J06 07 08 09 

ization A— 



Research Methods .... Questionnaires 01 02 03 04 05\06/)7 08 09 

Analytic Methods ... Bivariate Analysis ^ * 1 2 (z^ 4 5 8 9 

Analytic Focus ... Non-Qual1tat1ve i 10 8 9 

Reliability... Not reported * © 8 



& 



Validity n ^ J 1 )Z 8 

Reported _ 1 

Definition of Use ■ ' 1 (z) 3 , 8 9 

Conceptual , 

Object of Use... Qther 1 2 3 4 5 (T^B 9 



Practice Area and 

Population.. Nursing 



i 
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OVERALL POLICY IMPACT SCALE. 



Mark yan de Vail and Cheryl Bolas 



van de Vail, M. and C. Bolas. * Using. Social Policy Research 
for Reducing Social Problems: An Empirical Analysis of Struc- 
ture and Functions," Journal of Applied Behavioral Science, 
Vol. 18 (1982): 49-67: ~ : 

The-purpose of the overall Policy Impact Scale is to assess 
the Impact of research on organizational decisionmaking. 



Policy impact is operationally defined as specific organf- 
zational decisions or measures that, according to the researcher 
and policy-maker result from the conclusions of social policy 
research. ' 



The Overall Policy Impact scores were constructed by sumning 
the answers jto the following questions addressed to the re- 
searchers and policy-makers in the interviews: 

1. Manifest impact: Are there any identifiable spinoffs in 
decisions or policy measures (describe in full) from the re- 
search project at the following steps of policy formation: 



STEP , \ ROLE WEIGHT 

(A) initiating a policy ja) advising 3 

(b) co-deciding 6 

(B) preparing a policy ja) advising 2 

A (b) co-deciding 4, 

(C) executing a poliqy (a) advising , 1 

J (b) co-deciding 2 



Step A occurs at higher hierarchical levels, step B at inter- 
mediate levels and step C at lower (supervisory) levels; advising 
has lower impact upon policies than co-deciding. For these 
reasons, steps A, B, and C were weighted 3% 2, and 1 respec- 
tively, and roles -a and b were weighted 1 and 2 respectively.- 
The summed weights ranged from 0 through 18 (van de Vail and 
King, 1973).. 

2. Stage Impact:* Are there any identifiable spinoffs 1n de- 
cisions or measures (describe in full) at the following stages 
of social policy research: (1) formulating the problem, (2) 
operational 1 zing the problem in terras of research procedures, 
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. (3) sampling and collecting of data, (4) analyzing the data, 
(J) informal discussions with the client, (6) informal interim 
reporting to client, (7) final reporting to client, (8) trans- 
lating the research results into policy recommendations, (9) 
follow-up of recommendations; (10) correcting or adjusting 
recommendations, (11) otherwise. The weights ranged from 
0 through 11. . 

3. Certainty impact: Are there any identifiable spinoffs in 
decisions or measures (describe in full) regarding: (1) per- 
ception of the problem, (2) explanation of the problems's 
underlying causes, (3) assessment of the problem's seriousness, 
(4) dissolution of the problem. Weights ranged from 0 through 4. 

4. Latent impact: Are there any identifiable spinoffs in de- 
cisions or measures (describe in full); e.g., (1) postponing 

a decision, (2) 'enhancing rank and file participation, (3) 
increasing awareness of the problem, (4) enhancing the policy 
maker's status, (5) establishing an information monopoly, (6) 
preventing policy error. Weights ranged from 0 through 6. 

Simulating these four indicators resulted in Overall Policy 
Impact-scores (OPI-4) for the sociaj researchers (SR/OPI-4) and 
the policy makers (PM/OPI-4) (51-52). 



Internal reliability : Using Cronbach's procedure for establishing 
internal consistent reliability, the resulting alpha = .61. 

Test-retest reliability : In addition to in-depth probing, impact 
scores were independently rated by social: researchers and policy 
makers. 

Content validity : To ensure content validity, open interviews 
were held with researchers and policy makers outside the three 
samples and used for constructing the semi structural interview 
schedules. 

Internal validity : To ensure internal validity,. possible dis- 
tortion due to respondents' selective retention had to be taken 
into acdbunt. Respondents were invited to copsult project records 
regarding, utilization in order to minimize selective retention. 

Convergent validity : To establish convergent discriminant 
validity of the combined Overall Policy Impact Scale, the fol- 
lowing correlations between the researchers OPI scores and the 
policy makers OPI scores were established: industrial and labor 
relations, r = .49;. regional and urban planning, r = .69; social 
welfare and public health, r = .52; the mean correlation of the 
three combined samples,, r = .57 (n=120) (53) . 

Pragmatic and Item to Total Validity were also established by 
the authors (53). 
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ADMINISTRATION : The authors conducted a retrospective survey of three indepen- ' 
dent samples of forty projects of "client-oriented" social re- 
search used in one of the following sectors of social policy 
making in the Netherlands: 1) industrial relations; 2) re- * 
gional and urban planning; 3) social welfare and public health. 
For analyzing the 120 research^ projects, a triangulated design 
was used combining data from different sources: a theoretical 
and methodological analysis of the research report; a semi- 
structured interview with the social researcher and a compar- 
able interview with the policy maker responsible for incorpor- 
ating the research results into policy measures. 



SOURCES : van de Vail, M, and C. Bolas. "External vs. Internal Social 

Policy Researchers." Knowledge : Creation , Di/fuston , * . 

Utilization , Vol . 2 (1981): 461-81" 

van de Vail, M. and C. Bolas. "Applied Social Science Discipline 
Research or Social Policy Research: The Emergence of a Pro- 
fessional Paradigm in Sociological Research." The American 
Sociologist , Vol. 15 (1980): 128-37. ' 

g van de Vail, M. and C. Bolas. "Policy Research as an Agent of 

Planned Social Intervention." Sociological Practice, 
Vol. 2 (1977): 77-95. 
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STUDY PROFILE 

DIMENSION ( DESCRIPTION _( CODE 

- A 

Unit of Analysis Organization 01 02 03(04)05 06 07 08 09 

Sampling Convenience @02 03 04 07 08 09 

Descriptive Case, 

Design Study-explicit theory 01(02;03 04 05 06 07 08 09 

Research Methods Multiple methods 01 02 03 04 05 06(57)08 09 

Analytic Methods Multivariate 1 2 3 (?) 5 8 9 

Analytic Focus Nonqual itative i 1 © 8 9 

Reliability Reported 1 (f) 8 

Validity Reported 1 © 8 

Definition of Use Instrumental (J) 2 3 8 9 

o 1 

Object of Use Research study 1 (2 s ; 3 4 5 6 8 9 

Practice Area and 
Population 
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INDIVIDUAL DIFFERENCES IN THE SOLVING OF 
SOCIAL SCIENCE PROBLEMS - 



James Voss, Sherman Tyler, Laurie Yengo 

Voss, James, Sherman Tyler and Laurie Yengo. "Individual 
Differences 1n tFe Solving of Social Science Problems," 
In R.F. Dillon and R.R. Schneck, (eds.)', Individual 
Differences in Cognition , New York: Academic Press, 
forthcomi ng. 

The protocol was developed 1n order to study Individual 
differences 1n problem-solving styles when faced with 
ill -structured problems. 



Variables measured in this protocol are the cognitive 
problem-solving styles of individuals. 4 These are 
operational ized by breaking down an open ended response to 
a series of problem-solving questions posed by researchers. 

Primary research was carried out 1n a — quasi -experimental 
procedure at the University gf Pittsburgh. The flrtst 
consisted of eighteen respondents— four faculty members 
whose field of expertise was the Soviet Union, —six under- 
graduates enrolled in a course on domestic foreign policy, 
four faculty members who had fields of expertise 1r\ another 
social science discipline, and four members of the Chemistry 
Department. Respondents were asked to answer this question: 

"You are the head of the Soviet Ministry of Agri- 
culture, and assume crop productivity has been low 
over several years. You now have the responsibility 
to increase crop production. How would you go about 
doing this?" ' 



The Research was carried out 1n the context of an 1 
Information-Processing model of problem solving. Individuals 
when -confronted with a problem, (almost always in a particular 
"context or task environment") will first construct a* 
representation of the "problem space" and establish one or 
more problem definitions. The primary Issue the researchers 
wished^ to address was the differences 1n problem-solving 
styles and strategies between "experts" -those who were 
familiar with the substantial knowledge of particular classes 
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of problems, and "novices"-those who had either little 
knowledge of the substantive area, or were not experienced 
at tackling 111 -structured social -science problems. 

In order to operational ize the concept, the researchers 
modified Toulmlns 1958 and 1979 Argumentation Model: 

D = datum 

C = claim 

W = warrant 

B = backing 

R = rebiAtal 

Q = qualifier 



Modifications consisted. of three qualifiers: 1) claims(C) 
foay also be used as a datum for subsequent claim (D-C— -D); 
2) Backings-may also serve as arguments ((B)-D-C); and 3) 
qualifiers and rebuttals may also be arguments. 




RELIABILITY/ 

VALIDITY : Interrater reliability for both Initial codings (determining 
— argument units and connections between them) exceeded .85.. 



ADMINISTRATION : Responses to the question given 1n "Description" above, 

usually 1n the form of narratives or discourses were 

recorded for further analysis. Protocols were divided 
Into basic "argument units" which were then classified Into 
one of Toulmlns argument categories (see "Development" above). 
Connections between the argument units were also examined 
and represented 1n graphs. 

Further analysis was of several types: 1) Warrants were 
examined and classified for each protocol, 2) graphic 
, representation of the protocols. were used to examine "top- 
level" nodes of the graph- the "essential arguments" each 
individual used, 3) measures of complexity of Individual 
argument chains were constructed, 4) types of Backing 
statements were developed: a) simple fact or logical 
prlndple(B); b) a simple argument (D-C); or c> an elaborate 
"argument chain (D-C-CD)-C-(D)-C...). 

A number of interesting findings came from the protocol 
analysis. 

1) Experts spent a relatively Targe proportion of the 
protocol a problem representation (±24%). 

2) Novices tended to spend little time dev£lop1ng problem 
representations, but instead proposed specific solutions 
to the problem. 
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3) Experts tended to propose relatively few (but abstract) 
solutions to the problem, but devoted multiple 

levels of backing (mean = 8.8) for each solution. 

4) Novices devoted considerate time to proposing numerous 
solutions with little backing (mean = 2.3). 



SOURCES: 



Simon, H.A. "Complexity and the Representation of Patterned 
Sequences of Symbols." Psychology Review . Vol. 79 
(1972) :369-382. 

"Information-Processing Theory of Problem Solving 

Skills." Cognitive Psychology in W.K. Estes (ed.), 
Handbook of Learning and Cognitive Processes : Human 
Information Processing : Volume Five ^ Hillsdale, New 
Jersey: Lawrence Erlbaura Associates, 1978. 

Toulmln, Stephen, The Uses of Argument . Cambridge: 
Cambridge University Press, 1958. 

\ 
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STUDY PROFILE - - ' 

DIMENSION DESCRIPTION ' - CODE 

Unit of. Analysis r ' (oi))2 03 04 05 06 07 08 09 

Individual T - 

Sampling n A 01/62\)3 04 07 08 09 

Purposive • \Zs 

Design Quasi-Experimental without Q1 02 Q3 04 (Q 06 07 08 09 

control groups and/or randomization 

' Research Methods Content analysis 01 Q2@04 05 06 07 08 09 

Analytic Methods , B1yariate ana1ys1$ 1 2@ 4 5 8 9' 

Analytic Focus Non-qual native " l Q 8 * 

l_s ■ 

Reliability..... Reporte<j * Ql 8 

Validity Not reported . 1 

Definition of Use Conceptual * 1 (D 3 8 9 

Object of Use. . * Study 

1.(2)3 '4 5 6 8 ,9 

Practice Area, and 

Population * General, Educators 

\ . - ■ - * i, 

K 
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MANAGERIAL USE OF EVALUATION FINDINGS 



Edward C. Weeks 



Weeks, E.C. "The Managerial Use of Evaluation Findings" 1n 
H.C. Schulberg and J.M. Jerrell, The Evaluator and Manage - " 
ment. Beverly Hills, CA: Sage,(1979):137-155. t 



Thf purpose of this study was to provide empirical data 
regarding the relationships between organizational location, 
methodological practices, and decision context and the 
x utilization of evaluation findings. . 



The self-administered questionnaire measured th^ee inde- 
pendent variables and their effect on the dependent variable, 
utilization: The organizational location of the evaluator- 
was measured through an index of five elements including 
the frequency of meetings between program personnel and , 
evaluators during th£ evaluation planning phase; during 
the conduct of the evaluation, etc. 

i 

The methodological practices factor was composed of seven 
elements including, research design employed, sampling 
procedures, etc. The decision context factor was composed 
of six elements including the number of decision participants, 
specificity of program goals, existency of organized con- 
stituencies. 

The dependent variable, utilization was assessed on an 
instrument which presented ten statements describing 
varying degrees to which evaluation findings were applied 
to specific program decisions. 



The instrument to measure utilization consists of ten 
statements describing varying degrees to which findings could 
be applied. The respondent^ were asked to Indicate which 
of the ten statements \ best described the degree to which 
the evaluation findings and recommendations were used in 
making program decisions. 

Some of the statements are listed below: 

* Most of the decision- participants probably didn't even 
read the report. Nobody\con$1dered that it could tell 
them much they didn't already know/ ' ~ 



\ 



\ 
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* The decision participants discussed the findings and from 
that the decisions emerged, It's hard to trace the 
evaluation's exact role, but 1t definitely had some 
Impact on the decisions. 

* Many of the recommendations were adopted and Implemented. 
, In that sense, the evaluation study effectively "made the 

program decisions. 



DEVELOPMENT: 



The final statements used on this Instrument used to measure 
utilization were drawn from a larger pool of Items after 
all items had been* ranked by a small number of program 
evaluators and managers. 



RELIABILITY/ 
VALIDITY: 



The reliability of the scale was tested through a test- 
retest approach" using a parallel form. The retest form 
was mailed to a subsarflple of respondents and contained 
the same basic statements worded in a more terse, morq 
direct manner. The zero-order correlation between thd 
two forms was .84 (n=21)., (147) 



ADMINISTRATION: 



Following initial telephone contact, questionnaires were 
sent to the seventy-six evaluators of program evaluations 
which met Specific criteria. Fifty-seven (75 percent) of 
the questionnaires were returned and analyzed. 
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I 

STUDY PROFILE 

DIMENSION DESCRIPTION CODE 

: 1 ; ; : 

Unit" of Analysis....!... _ , ' 01 02 03 04 05 0$/6t)08 09 

J T Other (managers) ^ . 

< - \ 1 ■ 

Sampling ." Census 01 02 03(o4)07-08 09 

'• • ggg^^g 01 02 03 04g)06 07 08 09 

Research Methods Questionnaire 01 02 03 04 0 s(o6)o7 08 09 

Analytic Methods........ Bivaria te Analysis 1 2 © 4 5 8 9 

Analytic Focus.... , Non-qualitative 1 1 Q 8 9 

Reliability Reported 

I © 8 

Validity Not reported 

(T)2 8' 

Definition of Use.. Instrumental (j) 2 3 8 9 

Object of Use Evaluation Reports ©. 2 3 4 5 6 8 9 

— » 

Practice Area and 

Population % Evaluations of social programs implemented at the 

local level in California 

i 



- if- 



RESEARCH USABILITY STUDY * 



CaroivJ^iss and Michael J. Bucuvalas. 

Weiss, C.H. and M.J. Bucuvalas. Social Science Research 
an d^Deci s^xu^Maki ng . New York: Columbia, University Press, 



The purpose of this study was to assess the characteristics 
of social science research studies that make them useful 
for decision making. 



The dependent variables were two measures of usefulness. 
The independent variables were twenty-nine descriptors 
of research characteristics which can be loosely grouped 
/Into five categories: 1) clinical relevance; 2) research 
quality; 3) conformity to u$er expectations is 4) action- * 
oriented characteristics; 5) assumption challenging inform 
atlon. In addition various characteristics about the 
respondent were measured (e.g., highest degree, political 
orientation, years experience, etc.). 



Jhe respondents were asked to read two research reports 
which were summarized in the form of two page abstracts. 
They were then asked to rate each one of the twenty-nine 
descriptors of research characteristics, Indicating the 
extent to which, each descriptor applied to the study on 
the following 5-po1nt scale. 

1 . 2 3 4 5 8 9* 

J 1 I 1 * I L 

To a Largely Somewhat Slightly Not Don't. N/A 
great at Know 

extent all 



The instrument also contains a number of open-ended items 
designed to probe the conscious and unconscious use of 
research in the respondent! work, to determine the nature 
of information sources used, and the respondent's views 
of the appropriateness of research for various aspects 
of decisionmaking. 



Not reported: 



RELIABILITY/ 

VALIDITY : Not reported^— 



ADMINISTRATION : A stratified sample of 250 key mental health decision makers 

— and researchers was drayfn from the federrl , state gmd 

local levels from ten different states. j\ he personal 
Interviews were conducted by trained Interviewers, The 
supervision of the field work was intensive. Eighty-two 
percent of all attempted Interviews were^completed, and 
90 percent oFall attempted Interviews with elteible 
respondents were completed. m 



SOURCES : Weiss, C.H. and M.J. Bucuvalas. "*ruth Tests and Utility 
1 Tests: Decision Makers 1 Frames of Reference for Social 

Science Research". American Sociological Review , 45 

(April 1980):302-13. 
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STUDY PROFILE 



DIMENSION DESCRIPTION CODE- 

, ' Other (social science /*\ 

Unit of Analysis research studies) 01 02 03 04 05 06 VQ7)08 09 

Sampling Purposive 1 01^2)03 04 07 08 09 

Design. . . Uncodabl e 01 02 03 04 05 06 07 08 @ 

y — * 

Research Methods Interview 01 02 03 04 @ 06 07 08 09 



Analytic Methods Multivariate 1 2 3 © 5 8 9 



Analytic Focus Non-qualitative i 1 \2) 8 9 



Reliability (1) 2 8 

Not reported * 

Validity f)2 8 

Not reported 

\ 

Definition of Use Conceptual 

1 (£)Z 8 9 



Object of Use Uncodable 1 2 3 4 5 6 s(T) 



Practice Area and 

Population Mental Health, Policy Makers at the Federal, 

State and Local Levels 



If 5 



EMBEDDED FIGURES TEST 



Herman A. Witkin 



WitJdn, Herman A. Consulting Psychologists Press, Int. 
577 College Avenue, Palo Alto, CA 94306. 



The. Embedded Figures Test is designed to measure the degree 
of "field independence" an individual process. Field 
Independence is the degree. to which an individual exper- , 
iences "... his surroundings analytically, with objects 
experienced as discrete from their backgrounds." 



The Embedded Figures Test measures field-independence 
and field dependence. Witkin defines this dimension of 
perceptual functioning: The person witha more fiel^l- 
independent way of perceiving tends to experience his 
surroundings analytically, while the person with a more 
field-dependent way of perceiving tends to experience his 
surroundings in a relatively global fashion. 

■* 

The EFT consists of twenty-four complex, colored figures, 
in each of which one of eight simple geometric figures is, 
embedded. > The eight simple figures are also printed on 
separate cards so that they car) be presented to the re- 
spondent independent of the embedding context. A shortened 
form using only the first twelve items of the series has 
been shown to correlate in the mid-90 1 s with the total test. 
In addition, a group form of the test has been copyrighted 
by v the Educational Testing Service. The group form contains 
sixteen figures, with the complex figures falling on one 
page and the simple figures on the other. 4 Special instruct- 
ions for <^fildren are also available. 



The impetus for Witkin's adaptation of Gottschaldt' s 
figures was his dissatisfaction with the interpretations 
offered by the universal visual Gestalt view of the part- 
whole relation, or Gibson's view of the efforts of postural 
factors in explaining the perceptual property of "uprightness 
Witkin Instead developed the ideas of field-independence 
and field-dependence to account for differences in indivi- 
dual perceptions, and devised the test to measure these 
cognitive phenomenon. 
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RELIABILITY/ 
VALIDITY: 



Five studies of the reliability of the 24-1tem test have 
shown a median coefficient of .92, with a three-year 
test-reteit reliability of .89 for both young £dult men 
and women. The corrected odd-evetj reliability of the short 
form 1s .88. 

As to construct validity, Longeneeker (1956) compared the 
EFT with twp forms of the Holtzman Form-Recognition Test 
(Holtzman, 1955) to test whether a distinction exists 
between embedding fields and merely distracting fields. 
These correlated at .54 and .46 with EFT. Concurrent 
validity 1s suggested by the correlations of .46, .77 
and .69 1n three separate comparisons of the EFT with 
the Thurstone Gottschaldt test of flexibility and closure. 



ADMINISTRATION: 



The 24-1tem test 1s administered by the experimenter, who 
first shows the subject a complex figure for fifteen 
seconds and asks him to describe its overall pattern. Next 
he presents the simple figure as it 1s embedded in the complex 
one. The simple figure must be found in -the upright position 
and 1n the size 1t appears above. The time 1s noted for 
the subject to trace the simple figure within the complex 
one, which 1s his score. The time limit for each of the 
- twenty-four Items 1s five minutes; If he cannot find this 
figure, the time 1s noted and marked as Incorrect, tFte 
watch is started up from that time and the subject continues 
to try 'to find the simple figure. The fihort form of 
twelve items has a three minute time Vtmlt per item, 
while a group form imposes a total time limit of fifteen 
minutes for sixteen figures. For group form, the response 
format is a multiple choice among five alternative simple 
figures* The score on the group form 'is the number of simple 
figures successfully found within the fifteen-minute time 
limit. High mean scores indicate field dependence; low mean 
scores indicate field Independence. 



SOURCES : . Witkln, H.A. "Individual Differences 1n Care of Perception 

of Embedded Figures." Journal of Personality , Vol. 19, 

(1950): 1-15. 

Wltkin, H.A. "Cognitive Development and The Growth of 
Personality." Acta Psychol oglca , Vol. 18 (1961) : 245-57. 

Ho"jtzman, W.H. Tentative Manual : The Holtzman Form- 
Recognition Test . University ofTexas, 1955 (mimeo} . 



Witkln. H.A., R.B. 
and S.A. Karp. 
. of Development . 



Dyk, H.F. Faterson, D.R. Goodenough, 
Psychological Differentiation : Studies 
New York: Wiley, 1962. 
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ORGANIZATIONS COLLABORATING TO IMPROVE EDUCATIONAL PRACTICE" 



AUTHOR : % Robert K. Yin and Margaret K. Gwaltney 



AVAIL ABILITY: Yin, R.K. and M.K. Gwaltney. Organizations Collaborating to 

: : — : Improve Educational Practice , Cambridge, MA! Abt Associates, 

1981. 



PURPOSE : The purpose of this study was to advance the understanding 

of how and why interorganlzational collaboration produces^ * 

positive knowledge utilization outcomes. 



VARIA BLES: The study primarily focused on how different interorganlzational 
— arrangements worked successfully to provide knowledge utiliza- 
tion services. 

Three types of outcomes and eight types of explanations were 
examined. The outcomes Included: goods and services (e.g., 
product catalogs, workshops and conferences); utilization 
outcomes (e.g., Initiation of a planning or assessment activity); 
a change 1n educational practice (e.g., innovation) ; dysfunc- 
tional outcomes— the costs of collaboration (e.g., the added 
time needed to reach a decision because more participants must 
be consulted, the confusion of responsibilities that may be 
created by an interorganlzation arrangement). 

The explanations .include (at the organizational level ) mutual * 
exchanges, access to external funds, mandates to collaborate, 
formal agreements, and conflict mediation as well as (at the 
interpersonal level) self-fulfillment and career advancement (14). 



DESCRIPTION : This case study focused on three Regional Educational Agency 

arrangements selected based on several criteria including 

successful operation for a number of years. The three differed 
1n several ways— the student population served (urban, suburban, 
rural) and, 1n addition, each arrangement existed within a 
different state structure and legal mandate. Each of the (REA) 
arrangements was visited twice and each provided numerous docu- 
ments and records about its activities. 



DEVELOPMENT The inroediate roots of this study come from experience with the 

ABT Associates R&D utilization program. The authors examined 

various aspects of this program including the role of regional 
educational agencies. The authors were interested in determining 
whether these regional educational agencies also provided longer 
lasting, more significant lessons in Improving interorganlzational 
collaboration. 

ERIC 



for each case study, data were collected about four activities: 
1) staff development' services; 2) a linker assistance service; 
3) an Information retrieval service; 4) any broader organiza- 
tional Issues that appeared to affect these three services. 
The first three were selected because they represented the 
basid way 1n which information can be transferred. The fourth 
activity was selected because 1t sets the first three activities 
w1th1n\a context that 1s potentially relevant for explaining the 
operation' and outcomes of the other three activities. 
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STUDY PROFILE 

— * 

r 

DIMENSION DESCRIPTION CODE 

' , ■ 

„■ _ . , . Regional Educational '/"N 

Unit of Analysis ^l ncy Arranganen ts 01 02 03(65)05 06 07 08 09 

Sampling Purposive 01 @ 03 04 07 08 09 

Design Case Study 01@)03 04 05 06 07 08 09 

Research Methods Multiple methods 01 02 03 04 05 06(57)08 09 

Analytic Methods Ratings 1 (5) 3 4 5 8 9 

Analytic Focus Noncjual itative 1 1 (2) 8 9 

Reliability Not reported Q) 2 8 

Validity Not reported ® 2 8 

0ef1n1t1on of Use Instrumental (?) 2 3 8 9 

Object of Use other 1 2 3 4 5 6 (§) 9 

Practice Area and 

Population Regional Educational Agencies, Policy-makers 

and practitioners. 
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A REVIEW OF CASE STUDIES OF TECHNOLOGICAL INNOVATIONS IN 
STATE AND LOCAL SERVICES 



AUTHOR : Robert K. Yin, Karen Heald, Mary Vogel , Patricia D. 
Fleischauer, Bruce Vladeck 



AVAILABILITY: 



Yin, Robert K. , Karen Heald, Mary Vogel , Patricia D. 
Fleischauer, Bruce Vladeck. A Review of Case Studies 
of Technological in State and Local Services . Santa 
Monica: The Rand Corporation, 1976. 



■ PURPOSE : The study focuses o'n 140 case studies of technological 

innovations in a variety of settings. The major goal of 

the study was to discover organizational and external 
conditions, as well as the characteristics of the in- 
novation itself which users perceived to be "critical 
to the innovative process." 

»> , 

VARIABLES : Four sets of factors 1mp1ng1nq on the Innovation process 

were considered variables: 1) characteristics of the 

Innovation Itself (hardware, etc), 2) characteristics 
\,*.of the Innovating agency (personnel attributes, etc.), ' 
\3) external conditions (general sodaland political 
conditions, role of professional organizations 1n 
"fostering an Innovative environment"), and 4) character- 
istics of the Implementation process ("a mix of character- 
istics of the Innovating agency--e*g. , participation of 
service practitioners in early planning- and commtment 
by top executives-as well as personal attributes of inno- 
vations or implementors) . 

* 

DESCRIPTION : Researchers utilized the case survey method to gather 1n- 
formation. This method Involves a checklist of closed- 
ended questions which "reader-analysts" answer in reviewing 
project final reports, agency-or-consultant-produced formal 
evaluations, or articles in journals. Generally, researchers 
utilized Snowball sampling 1n discovering and evaluating 
material for Inclusion in the sample. Criteria for in- 
clusion included the existence of a written report des- 
cribing "organizational events in a specific local agency," 
which described an organization's first use 0* a technical 
innovation ("a new machine, material, chemical, computer 
system, or quantitative analysis."). 

Exclusions were made if the report was generated through 
a trade journal (non-academic) or only as the result 
of a conference, or appeared before 1965. 
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DEVELOPMENT: 



The survey was Initiated out of a desire to assess critical 
characteristics 1n innovation adoption and Implementation. 
The major hypothesis of the study was that Implementation 
factors would be most critical. 

[This] hypothesis drew directly from the observed 
failure by two traditional Innovation and 
diffusion frameworks—the research, development 
and diffusion approach and the social Interaction 
approach— to account for Innovation 1n local 
organizations. (P. v11) 



RELIABILITY/ 
VALIDITY: 



Inter-rater reliability was calculated. Coefficient were: 

' RELIABILITY ANALYSIS OF @# CASE STUDIES 
BASED ON RATINGS OF TWO ANALYSIS 



Level, of 
Confidence* 



Percentage 
of 

Responses 



Mean 



Percentage of Agreement 



Standard Deviation 



Sure 
Not Sure 
Total 



74.4 

r 

25.6 
100.0 



77.1 
59.8 
71.9 



7.8 
15.7 
6.5 



i 



*"Not Sure" Includes those responses where one or both 
analysts were unsure. 



Taken from: A Review of Case Studies of Technological 
Innovations In State and local Services. 
The Rand Corporation ."TSTtTT 

No validity ratings were reported. 



ADMINISTRATION: 



One-hundred anct forty cases were examined using the case 
study method. ladh "reader-analyst" used a closed-ended 
question checklist (See "Development" above) to analyze 
the cases. For each case-study* analysts answered cffeckllst 
questions and gave a "level of confidence" rating 'sure 1 
or 'not sure 1 for each answer. 



SOURCES : Y1n f Robert K. and Douglas Yates. "Using the Case Survey 

Method to Analyze Policy Studies." Administrative 

Science Quarterly . Vol. 20, (September 1975) :371-8J. 

* « 

Zaltman, Gerald, et.al., Innovations and Organizations . 
Hew York: John Wiley and Sons, 1973. 
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jTUDY PROFILE 



DIMENSION DESCRIPTION CODE 

' ; : 

Unit of Analysis Organization 01 02 03 04 05 06 07 08 09 

Sampling Purposive 01 02 03 04 07 08 09 

Descriptive case study with 
Design Explicit Theory 01 02 03 04 05 06 07 08 09 

< ' 

Research Methods Content Analysis 01 02 03 04 05 06 07 08 09 



Analytic Methods BlvaHate Analysis 1 2 3 4 5 8 9 



Analytic Focus Non-qualitative 

Reliability Reported 



Validity Not reported 12 8 

, Definition of Use Instrumental 1 2 3 8 9 

1 : 

* Object of Use Evaluation 12345 6 89 

., , ! i 

Practice Area and * 

Population Local -Government, Policymakers, Practitioners 



2>Ki 



USE^OF MARKET RESEARCH 



188 



0 



AUTHOR: 



Gerald Zaltraan and Rohit Deshpande 



AVAILABILITY: 



PURPOSE: 



VARIABLES: 



Zaltman, and R. fleshpande. "The Use of Market Research: 
An Exploratory Study of Manager and Researcher Perspectives," 
Cambridge. MA: Marketing Science Institute. 1980. / 

\ 

This was an exploratory study designed to explore the factors 
which Influence the effective use of market research by 
market managers. / 

/. \ ' 

The major sets pf variables considered were use of research 
Information, organizational structure of managers 1 firms, 
stage of the product life cycle, characteristics of the 
research report, and extent of Interaction between managers 
and researchers. 



DESCRIPTION: 



Separate Instruments were used for managers and researchers, 
The questionnaire instruments were constructed by modifying 
questions used in personal interviews to take the form of 
closed structured items. After preliminary questionnaires 
asked respondents to focus on the most recently completed 
marketing research project with which they had been 
associated. 



DEVELOPMENT : 



The first stage of the study consisted of personal interviews 
with sixteen Individuals in seven firms (ten managers and 
six research suppliers). The sixteen persons were selected 
on a convenience basis from large organizations (all in the 
Fortune 500 group of corporations) and from leading advertising 
'and research agencies. 



The personal interviews provided considerable Insights Into 
issues to be explored in the mall questionnaires and also 
.a basis for interpreting some of the results obtained from 
the mail survey data. The questionnaire Instruments were 
constructed by modifying the personal interview questions 
to take the form of closed, structured queries. 



RELIABILITY/ 
VALIDITY: 



Reliability coefficients are reported for all variables scales 
used in the model. All but four of the coefficients are 
significant. 
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ADMINISTRATION: 



Two different types of validity were assessed. Inter-item ' 
index, validity was measured by correlating". "each item on an 
index with Jthe index itself (all correlations are significant 
at- 0.001 levels). Convergent validity was. measured by%or- 
relating selected indices with' other' items in the question- 
niares. , ■ — 

• . ■ ' j 

The questionnaires were mailed to two separate cross** sectional 
samples of managers and researchers. The mail survey 
produced 176' usable responds out of an eligible sampTe of 
397 (44.355 response ratej^the final response rate for 
managers was 86 out of 249"eligible respondents (35t) and 
for researchers :90 out of 148 eligible respondents (61%). 
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'• ' STUDY PROFILE '^V 
DIMENSION _^ DESCRIPTION : CODE 

I • r - 

ft 

Unit of Analysis other ( M rket „.n.9ers) 01 02 03 04 05 06 (65) 08 09 

Sarapl1ng Random 01 02 ®° 4 07 08 09 

v Cross-sectional with , -~ v , i 

Design statistical controls 01 02 03^04)05 06 07 08 09 7 , 

. Research Methods.* Questionnaire 01 02 03 04 05(j6)07 08 09 

Analytic Methods Multivariate 1 2 3 © 5 8 9 

Analytic Focus Non-qualitative i 1 (2) 8 9 • ' 

Reliability 1(2)* 

Reported 

Validity R e p 0r ted 

Definition of Use Instrumental (T) 2 3 8 9 

Object of Use.... Market research report 3 4 5 6 8 9 

, ' 

Practice Area affd > 
Population Marketing, managers and researchers 
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APPENDICES 




A SAMPLE PROCEDURE ABSTRACT FORM 
B SAMPLE STUDY PROFILE 

C FREQUENCY DISTRIBUTIONS FOR STUDY PROFILES 
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APPENDIX A 
PROCEDURE ABSTRACT FORM 

' AUTHOR: 



AVAILABILITY :" 
PURPOSE : 

r- 



VARIABLES: 



DESCRIPTION: 



DEVELOPMENT : 



RELIABILITY/ 
VALIDITY: 



ADMINISTRATION: 



SOURCES: 
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APPENDIX B 
STUDY PROFILE 

DIMENSION DESCRIPTION CODE - 

; Unit of Analysis ' 01 02 Q3 04 05 06 07 08 09 

Sampling 01 02 03 04 07 08 09 

1 : 

Design 01 02 03 04 05 06 07 08 09 

Research Methods Ol 02 Q3\04 05 06 07 08 09 

Analytic Methods 1 2 3 4 5 8 9 

Analytic Focus * i 1 2 8 9 

Reliability. > 12 8 



Validity 1 2 8 

Definition of Use....... 1 2 3 8 9 

Object of Use 1234568 <9 

1 ■ —! 

Practice Area and 

Population 

) 

o ^ 

i 

s 



APPENDIX C 

FREQUENCY DISTRIBUTIONS FOR STUDY PROFILES 
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Absolute 

.1^ 
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1 . y 

• i • > * C 
i . 0 ? ? 



7. 

Total 



Std err 
"lscirn car 



1 

17 

2 

0 D 



( h ) 

4 4.0 
3.4 
5.2 

13.3 

2^.3 
3.4 
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I re 1 
< * ) 

•i6. 1 

' 3.6 

5. 1 

.14.3 

30.4 ■ 

:•: i a s l n j 

-10 0.0 



0 .350 

2.61 'A 

0 . 4o*9 X 
7. O00' 



4e'i 1 dfl 

/•iri.iriC*? 

n" 1 II q 0 



1 : 



!».'.#« C 



(Cro.-ti o.l cat* = 27- K U'.j-:w ) 



w'w.FLr." Jr. sic. 



21u 



9 

ERIC 



193 



f re.j 
' ( *• ) 

1 o . <i 

50.0 

5 j . t 

•j * . q 

1 0 0 . c 



2 . 50'. 
'j . 3*1 
6 . vOO 



; 7 - A u 



.SO i- P 

i i c 



i 

.'. i' 

I 

I 
I 

.SO 

I 

I 

i 

C ^ 

i 
I 

I 

La 

I 

1 

I 

I 
I 

1 

.'tc 

r * 

lERLC 



194 



1 Tc 3 



i r. i j 

t r i.j 

( •' ) 

i i . t 



Til 
C - ) 



i j. 

















• V 














~) 










: 7 i . c 










2 




3 


.4 • i a 




^ j «j . 








Total 






100 


. 0 10 


j. 0 








.;':7 


: err 


'j • 






: i .4n 




. 0 1 0 
















( i iTiC 








I 
















J . 0 ) o 




1 


• I", J 






0 u 0 






- 










' \ i r> w> i ivj c t 




2 
















. J . j Jrt . h 




















r i on oitn.r 2^ 


- rt U 1 - 0 













c 

J -i 





> l<iK 


l 




Absolute 


I r 
( 


o ) 


. r r t - j 
' .( i ) 


( 


C « 




A i - 1 i 


1 . 








i.?.3 


i J • 5 


C " 
. i 






• 


i : 


17 


') 

• «- 








w> L) f CT 


-;.r i • 










: ) . 3 


•1 o . 2 


-c* 


»*^r-: CI 


— j i * r 


't • 


'i 




. l > 


iO.S 


> - • n 


A *~ * * 








17 




. 0 


; ; . T 


1 3 >J . j 




mil: 




c • 




3 


. 1 


•* 1S3 in j 




1 






'j • 


17 




. 3 




1)3.', 








Total 




luv 


.u 


: j o . c 








b . 00 C 




1.5^'J 




4i Jl 

1 


rj n c e 


3. ;7l. 
2. 30 v 



.7-.Su 



211 



I 

I 
I 



t 'J .« * 



r 



^0 




n 



V .» I 1 W J ? jl * 



f3 . V 6 C 
. . i 1 5 



1 1 In j c -i .5 - : 



i 

I 
I 

I 
I 
I 

R 1 1' -a o t y 1 o r t* 1 c o j <i 





A n o 1 ii t e 
I r h '! 


r re j 


-\ J J 'J j 

( i 


1 




z • 




3. « 


^ • 








iO * 


17.2 


17. 


? 


? J • 7 


6 . 


5 


3.0 


K 




? i.3 




.H 


•H . 3 


n i . 




7 /.o 


7.. 


11 


22. 4 






1 ) ) . * 


To td I 


v> 


1 C 3 . 0 


1 , ). 






r, <P»Rfc < 5*5 


f\ 1 * 

✓ • I 3 1 

i * 7 ; 

- i • ; a 3 


> iM 
I h i 1 7 


• r:c e 




'j • j 

*-•-». f 
0 • J'Ju 



«. 7- «u # 



1 - 



• w.T'- ("re it ion c^t - = 

n L rl.nLl I iC ? Mi, ^ 



A l ) s c 1 j t ti t r - \ 



l r c ] 
i 5 



r 



:RLC 



( o ) 

12. i 
2 7 . c 

212 



! J 'J i t > c 
( I ) 
1 J . "J 
j0.2 



CM.JI 
f *■ 

( >. ) 

1 > 



7- * 1 



196 



LT'*- I ? T; 



i i 



V 



of) 



i . 7 1 > 



- : o * n s ' 



"1.7 

1 • J . i 
ju J • \, 





i v l . 


/i - i % » r\ 






,}. 


1 


4 »i j 1 -a p . 


2. 


73 j 


1 . 






1. 


121 


-•">. 


CV-t 7 


,< jn^y 


t . 


j C s. 


■> . 


c o ; 









1 1 * 



) J 



» - • » 1 1 .) w J «/ V 

( r » j 1 1 o rj u *i r r 



,\ ^ b o 1 u t e 
tre 



f r e j t r e i 



• . /.«l/l . I 1 



e f; 

I t o 5i 1 
i I xMW 

i 1 v- C,:ia: 



1.7^ 

i r; 



re.! 

.iX T1 




* r * 



; 'in 

1 , <*. ' ■ 

^ • U v ^ 



ERIC j 



213 



*7- \u- 



-7-Au; 



1 
1 

I 

I 



f '■ L T - ' . L I ? V 



197 



ci cf 



r\ V w C I 'J t < 



iBcrr; 

Kill to:, l 

t" 

v i : i .. c. 



1 . -p 1 

1 . : 



To to 1 



1 y x «r u ? 



( i ) 



•43.1 

ICC • j 



( i ; 

.1.1 

1 'j 0 . J 



■ [e, 
( * ) 



j . t c c 



' e j i i r 



L. 37, 
1 * j )l 



I 

I 
I 

I 

1 
t 

t ' 

I 
I 



I L - 1 A ' _ fl. 



1 ^ 



1 . VP s i i ; 
( r: r r tin: 



7-; ij.j-o: j 









V a 1 .i 1 1 j e ; 


< i J 'J 3 t G 




•J U 






A s t o 1 u t e 


i r 






re ^ 




C o u > 




( i ) 


( 4 ) 


( 


*- ) 










C "3 . i 




w . <! 




^ « 


s 


21 .0 


J 1 . * 










1 


1 .7 


I J > 1 P -j 


' *s 

A • ' 


0. u 




Tut,!! 




ICO. ■) 


1 0 o . o 






1 . :1 o 


rt i -Li 


v- . ' 1 2 






X • 




i • I c 


f I ; v.-v 


0 . - i, * 


/jri.i 


Pi ^ 


/ • 




. i'-<7 


" p. « » n .» r ? 


r \ . r i 






*l • 





I 
I 
I 

r l 

I 



u 11 .nil "> 

i zj. i c a r,ts 



7-Au- 



vr.;?* 'o t c., i 

ril» Ji,:;*'-':. (Tre.it i or; u.it* 1 = rT-AUj-. J 

■ERIC 



21 a 



.7- 5 u 



it * j.h / 1 



r ^T7/ L 



f' p * - f , 



.vie r-os is 
J i j j ^ J 



1 

I 
I 
I 

1 

I 

1 
1 

I 
I 

I 

I 
I 

i 
t 

i 

I 
i 



198 



J • T - T 



u 0 .4 ? 



i r s 



• La U / > 

( 6 ) 
' i J • ? 

4.4. : 



* j 'J i t-i 

( « ; 

71' 



J nr. 
( -i ) 





J • 


2 




3.-, 




i . 


t 










3 . i 


"nun j 








Tote! 




. j 


1 • V 










J t 'i -rr 


• 


•;7j 




i j i j n 


i . 


^ 


. '.'0 ^ 






r -,3 


t 




J • 


? fi : 


. J?C 




i • 


47? 


1 


in 


it* 


> > 






* . 


V/' 1, 










3 * 


' i s ir.*7 »: j sy . 















* '.' i-^z ;, ! s . 



1 1 1 I M 




' I : n .a 1 ' < , 

'LiC"- J-J7. - f I - 

: J "i u- r : ■ ; r : • L I :. ,n . 



0 ^ ** 

1 . 



1 1 



i 

ot.u 



£ r -t j 
( i ) 

1 3. j 

L • .4 

.3 

1C .1 

i 3 . l i , 

3. « 

I 



*. J 'J o t ■• ^ Cur.: 
tr«;i f r t; 



( J ) 
1 ).■-, 

i ! . <) 
i i.i 

i ; . 5 



( * ) 
1 y . 
j 1 • a 
> "j . > 
•3 1. . J 
I ) u . ^ 

: ') -j . „ 



:7-! U 



21b 



lllllltjlM 



t ; 1 ?v ' 



i .V/-; 



199 



< ^ r4 r» c } 

.« ir; to 



3.4' 



\ 



IERLC 



216 



